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Raytheon  Technical  Services  Company  (Contract  F41624-95-C-501 1).  The  objectives  of  this 
contract  are  to  provide  behavioral  research  support  in  the  areas  of  combat  mission  training  and 
mission  rehearsal.  The  contract  effort  was  conducted  under  Work  Unit  1 123-B2-06,  Aircrew 
Training  Technical  Support.  The  Laboratory  contract  monitor  is  Mr  Jay  Carroll.  The  in-house 
work  was  conducted  under  Work  Unit  1 123-B1-02;  Special  Operations  Aircrew  Training  and 
Mission  Preparation  Research.  The  technical  monitor  is  Dr.  Robert  T.  Nullmeyer. 
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Squadrons  for  their  support  in  scheduling  and  participating  in  this  study.  Finally,  we  want  to 
thank  Major  Ned  Linch  for  being  the  wing  point  of  contact  to  set  up  the  interviews. 


SURVEY  OF  COCKPIT/CREW  RESOURCE  MANAGEMENT 

FOR  F-16  PILOTS 


INTRODUCTION 

The  goals  of  U.S.  Air  Force  Cockpit/Crew  Resource  Management  (CRM)  training,  as 
defined  by  Air  Force  Instruction  11-290,  Cockpit/Crew  Resource  Management  Training 
Program,  are  to  maximize  operational  effectiveness  and  combat  capability  and  preserve 
personnel  and  material  resources.  CRM  training  provides  crewmembers  with  performance¬ 
enhancing  knowledge  and  skills  tailored  to  fit  the  unique  characteristics  of  each  primary  mission 
and  covers  six  core  behaviors:  Situational  awareness,  crew  coordination/flight  integrity, 
communication,  risk  management/decision  making,  task  management,  and  mission 
planning/debrief.  While  CRM  training  appears  to  be  readily  accepted  by  Air  Force  aviators  who 
fly  multicrew  aircraft,  including  two-seat  fighter  aircraft,  there  are  variances  among  single-seat 
fighter  pilots  in  the  perception  of  the  applicability  of  CRM  training  in  their  environment.  These 
perceptual  differences  are  highlighted  by  the  fact  that  single-seat  fighter  aircraft  do  not  have 
"crews”  per  se  but  rather  operate  in  "flights"  of  individual  aircraft,  each  with  a  single  pilot,  to 
accomplish  their  mission  through  a  mutually  supporting  effort. 

This  inquiry  was  undertaken  to  examine  how  F-16  fighter  pilots  viewed  the  Air  Force 
emphasis  on  CRM,  the  breadth  and  depth  of  CRM  skills  and  behaviors  and  their  applicability  to 
the  "single-seat  fighter  community,"  potential  changes  to  CRM  training,  and  the  pilots’  dominant 
learning  style.  This  qualitative  research  was  conducted  with  the.  56th  Fighter  Wing  at  Luke  Air 
Force  Base  AZ,  where  extensive  interviews  were  conducted  with  36  F-16  pilots  from  three 
different  squadrons.  Pilots  were  also  queried  about  their  preferred  modes  of  learning.  The 
conclusions  and  recommendations  in  this  study  highlight  what  potential  enhancements  and 
modifications  could  be  made  to  CRM  training  programs  to  help  maximize  the  acceptance  of  the 
training,  as  well  as  improve  the  effectiveness  of  single-seat  fighter  pilots'  CRM  skills  within  their 
flights  and  mixed-force  operations. 


Background 

Cockpit/Crew  Resource  Management  for  single-seat  fighter  aircraft  is  a  relatively  new 
approach  to  solving  an  old  problem:  How  can  aviators  make  the  most  effective  use  of  all 
available  resources— physical  assets  and  personnel.  CRM  training  involves  a  focus  on  aviation 
human  factors.  Inadequate  training  in  aviation  human  factors,  that  is,  incomplete  situational 
awareness;  fixation  and  distraction;  weak  inter-  and  intra-flight  communication;  poor  crew 
coordination  and  flight  integrity/discipline;  and  inadequate  mission  planning/debriefing,  risk 
management/decision-making,  and  task  management,  has  caused  numerous  lost  lives  and 
accidents  which  could  have  been  prevented. 

In  the  early  1980s,  Cockpit  Resource  Management  was  introduced  into  the  aviation  arena,  at 
first  mostly  in  the  airlines.  Commercial-aviation  CRM  started  out  centering  on  personality 
variables,  attitudes,  and  management  styles.  Noted  aviation  human  factors  researcher  Robert 
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Helmreich  characterizes  the  development  of  CRM  as  occurring  over  five  generations,  to  date. 
Prior  to  1980,  crew  coordination  was  traditionally  focused  on  the  individual  pilot  and  reviewing 
accident  reports  for  ways  to  improve  flight  operations.  In  the  early  1980s,  first-generation 
“Cockpit  Resource  Management”  started  a  shift  toward  measuring  the  pilot’s  “attitudes”  against 
an  ideal  standard,  as  an  improvement  vehicle.  By  the  mid-1980s,  second-generation  CRM 
brought  the  entire  flight  deck  into  the  process  with  an  emphasis  on  cockpit  group  dynamics, 
along  with  a  name  change  to  “Crew  Resource  Management.”  During  the  early  1990s,  third- 
generation  CRM  incorporated  all  of  the  organizational  personnel  into  the  process  (flight  crews 
and  maintenance,  etc.),  while  stressing  team-building  skills  to  enhance  performance.  By  the 
mid-1990s,  fourth-generation  CRM  widened  the  focus  to  bring  the  entire  team  under  the  CRM 
umbrella  (pilots,  support,  maintenance,  air  traffic  controllers,  etc.)  and  to  emphasize  procedural 
integration  by  adding  specific  “behaviors”  to  their  checklists.  The  goaf  of  fourth-generation 
CRM  was  to  assure  that  decisions  made  and  actions  taken,  even  in  nonstandard  situations,  were 
the  results  of  making  human  factors  and  CRM  an  integral  part  of  all  flight  training.  (Personal 
communication  with  Helmreich,  in  Spiker,  Tourville,  Silverman,  &  Nullmeyer,  1996;  Helmreich, 
Merritt,  &  Wilhelm,  1999).  The  breadth  of  CRM  training  has  continually  grown  to  include 
communication,  situation  awareness,  decision-making,  and  other  human  factors.  A  recent  aim  of 
CRM  training  has  been  to  examine  error  chains  and  explore  strategies  to  manage  error.  The  fifth 
generation  of  CRM  unfolded  in  the  late  1 990s  for  the  new  millennium  with  the  perspective  that 
since  “human  error”  is  inevitable;  it  is  therefore  a  valuable  source  of  information.  Accordingly, 
CRM  can  be  viewed  as  exercising  a  set  of  error  management  countermeasures:  first,  to  avoid  the 
error;  second,  to  trap  an  error  before  it  can  be  committed;  and  third,  to  mitigate  the  consequences 
of  an  error  that  does  occur  and  was  not  trapped  (Helmreich,  Merritt,  &  Wilhelm,  1999).  Current 
research  also  accentuates  that  learning  must  be  a  continuous  process  over  time  to  be  effective 
and  stresses  the  importance  of  incorporating  cognitive  psychology  models  in  team  building 
training  programs  (Ilgen,  1999). 


In  the  U.S.  military,  the  scope  of  CRM  evolved  from  an  emphasis  on  the  aircraft 
commander,  to  training  that  is  applicable  to  active  inclusion  of  all  flight  and  crewmembers.  The 
Air  Force  embraced  CRM  and  in  1994  issued  Air  Force  Instruction  36-2243,  Cockpit/Crew 
Resource  Management  Program,  which  established  the  requirement  for  developing  and 
managing  tailored,  mission-specific  CRM  training.  In  1998,  Air  Force  Instruction  (AFI)  1 1-290 
replaced  AFI  36-2243  as  the  Air  Force’s  instruction  on  CRM  for  flying  operations.  Each  Air 
Force  major  command  is  responsible  for  supplementing  the  Air  Force  instruction  by  creating  its 
own  specific  training  for  its  individual  aircraft  and  missions.  Air  Combat  Command  (ACC),  Air 
Education  and  Training  Command  (AETC),  the  overseas  commands-United  States  Air  Forces 
Europe  (USAFE)  and  Pacific  Air  Forces  (PACAF),  and  the  Air  Force  Reserve  and  Air  National 
Guard  fighter  units  all  have  significant  fighter  CRM  training  requirements  in  the  Combat  Air 
Forces  (CAF).  This  situation  creates  the  potential  for  variances  in  the  CRM  training  received  in 
the  different  major  commands. 


Specific  Research  Objectives 

The  primary  purpose  of  this  research  was  to  gain  an  understanding  of  single-seat  fighter 
pilots  attitudes  toward  CRM  training,  with  a  long-term  objective  of  using  the  data  to  enhance 
CRM  training  for  the  fighter  community.  Single-seat  fighter  pilots’  attitudes  toward  CRM  and 
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learning  styles  were  explored  by  interviewing  a  representative  sample  of  F-16  pilots  at  Luke 
AFB. 

The  overarching  research  question  in  this  inquiry  was  “What  enhancements  to  single-seat 
fighter  pilot  Cockpit/Crew  Resource  Management  training  programs  could  improve  the  pilots’ 
understanding  and  acceptance  of  CRM  training,  with  the  goal  of  improving  their  long-term 
retention  and  application  of  CRM  training?'1 

The  analysis  of  the  data  from  the  interviews  also  included  an  examination  of  the  perceived 
attitudes  and  needs  of  the  single-seat  pilots.  Additionally,  a  component  of  the  inquiry  was  to 
examine  the  learning  styles  of  the  F-16  pilots,  in  an  attempt  to  determine  the  most  reinforcing 
format  for  CRM  training.  Also,  a  composite,  integrated  CRM  training  model  is  suggested  to 
enhance  training  retention  and  application. 

Organization  of  Report 

The  remainder  of  this  report  is  presented  in  three  major  components.  The  first  component 
will  detail  the  methodology  used  to  conduct  the  qualitative  research  involving  F-16  pilot 
interviews.  The  second  section  examines  the  interview  findings,  and  includes  examples  of  the 
most  significant  interview  comments  that  fall  under  the  following  topical  subjects: 

1.  Formal  CRM  training.  Presence  or  absence  of  previous  CRM  training  is  explored,  along 
with  an  assessment  of  the  quality  of  training  (neutral  or  mixed  experience,  strongly  positive 
experience,  or  less  than  positive  experience)  of  selected  training  components. 

2.  Attitudes  toward  F-16  CRM  training  program.  Attitude  assessment  topics  include:  F-16 
pilots  who  feel  that  they  are  already  doing  CRM;  F-16  pilots  feel  that  they  are  being  “forced”  to 
do  CRM  training;  use  of  the  term  “CRM”  is  not  a  common  or  desired  point  of  reference  for  F-16 
pilots;  F-16  training  should  focus  on  single-seat,  not  multicrew  aircraft;  USAF  needs  to  show  that 
it  is  serious  about  CRM;  other  positive  responses,  and  other  negative  responses. 

3.  Suggestions  to  improve  CRM  training.  Respondents’  suggestions  are  categorized  under: 
Make  all  CRM  training  tactically  relevant,  make  CRM  training  applicable  to  single-seat  fighter 
pilots,  consider  using  another  term  other  than  “CRM”  for  single-seat  fighters,  assure  course 
delivery  is  applicable  to  audience,  assure  concept  of  training  (format,  duration,  frequency,  etc.) 
enhances  learning  process,  leadership  must  support  training,  use  of  simulators  to  reinforce 
training,  do  not  force  training  to  be  accomplished,  CRM  instructors  must  have  credibility,  and 
general  comments. 

The  third  component  provides  overall  interpretations  and  conclusions  from  the  research,  and 
proposes  recommendations  for  enhancing  single-seat  fighter  Cockpit/Crew  Resource 
Management  training  programs. 

Appendix  A  details  each  of  the  respondents  by  an  identification  number.  Appendix  B 
outlines  the  respondents  by  pilot  category  (instructor,  student,  or  leader),  and  Appendix  C 
summarizes  the  respondents  by  their  experience  in  different  types  of  aircraft.  Appendix  D  is  the 
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Consent  Form  for  the  Interviews.  Appendix  E  is  the  Interview  Worksheet.  Appendix  F  contains 
the  complete  interview  responses— the  number  in  parentheses  following  the  comment  indicates 
which  respondent(s)  made  the  comment.  Appendix  G  contains  responses  to  other  interview 
questions  on  CRM  behaviors  and  single-seat  fighter  flight  operations  which  were  not  included  in 
their  entirety  in  the  findings  and  interpretation  sections  but  which  still  offer  a  rich  narrative  with 
which  to  examine  and  understand  single-seat  fighter  pilots’  attitudes  toward  Cockpit/Crew 
Resource  Management. 


METHODS 

Participants 

Extensive  interviews  were  conducted  with  36  F-16  pilots  from  three  different  squadrons  at 
the  56th  Fighter  Wing,  Luke  AFB,  AZ.  These  pilots  represented  three  groups:  F-16  instructor 
pilots  (n  =  14),  F-16  student  pilots  (n  =  11),  and  squadron  and  wing  leaders  (n  =  11). 

Interview  Content  and  Structure 

Interview  questions  (Appendix  E),  in  an  open-ended  format,  were  used  to  initiate  and 
facilitate  the  interview  process.  Questions  were  written  in  two  forms:  general  questions 
providing  the  overall  structure  for  the  interviews,  and  subquestions  to  narrow  the  focus  and  to 
provide  the  ability  to  probe  the  informants  responses  in  selected,  specific  areas.  Questions  were 
designed  to  support  anticipated  data  analyses  which  included:  (a)  “open  coding”  to  determine 
categories  based  on  the  responses  of  F-16  instructor  pilots,  F-16  student  pilots,  and  F-16  leaders; 
and  (b)  “axial  coding”,  which  looks  at  the  comparisons  and  interrelationships  of  the  F-16  pilots 
in  regard  to  CRM. 

The  research  employed  qualitative  research  techniques  where  the  common  respondent  topics 
and  links,  foundational  theory,  and  hypotheses  are  allowed  to  evolve  during  the  on-going  data 
collection  and  analysis  without  preconceived  hypotheses  (Creswell,  1994).  The  goal  was  to 
provide  behavioral  and  attitudinal  data  points  from  the  single-seat  fighter  community  that  would 
facilitate  modifications  to  enhance  CRM  training. 

Participants  were  first  asked  to  provide  demographic  information  (flight  experience,  position 
within  the  Air  Force,  etc.).  Pilots  were  then  queried  about  their  preferred  modes  of  learning,  with 
an  objective  of  improving  academic,  simulator,  and  flight  training.  CRM-specific  questions 
addressed  exposure  to  formal  CRM  training  and  reactions  to  that  training,  integration  of  CRM 
principles  into  everyday  operations,  instructors’  attitudes  of  students’  understanding  of  CRM  or 
students’  attitudes  regarding  instructors’  and  evaluators’  use  of  CRM,  and  recommendations  for 
improving  CRM  training  for  F-16  pilots. 

Data  Collection  Procedures 

Each  interview  lasted  approximately  one  hour  and  involved  two  researchers.  The  principal 
investigator  was  a  former  F-16  pilot  and  a  former  fighter  wing  commander.  This  researcher, 
who  is  on  the  faculty  at  Arizona  State  University,  conducted  each  of  the  36  interviews.  In  each 
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interview,  the  principle  investigator  was  supported  by  one  of  three  other  researchers.  Most 
interviews  were  supported  by  one  of  two  Air  Force  Research  Laboratory  members,  either  an 
active  duty  captain  who  was  formerly  an  instructor  navigator,  or  an  Air  Force  civilian  with 
substantial  field  research  experience.  For  three  of  the  interviews,  a  graduate  student  from  the 
Arizona  State  University  aviation  program  provided  data  collection  support.  The  principle 
researcher  asked  all  interview  questions  while  he  and  the  second  researcher  recorded  responses. 
Some  responses  were  recorded  verbatim,  while  others  were  summarized. 

The  researchers  had  the  support  of  the  senior  wing  leaders  who  approved  access  to  the 
population  samples  to  be  studied.  Steps  were  taken  to  obtain  permission  from  the  F-16  wing,  so 
that  the  rights  of  the  human  subjects  were  protected  (Appendix  D).  The  researchers  recognized 
the  sensitive  nature  of  maintaining  confidentiality  of  the  data,  preserving  the  anonymity  of  the 
informants,  and  using  the  research  only  for  its  intended  purposes  (Creswell,  1994). 

Interviews  were  conducted  in  the  fall  and  winter  of  1998  and  the  spring  of  1999.  Interview 
sessions  were  continued  to  the  point  where  there  was  a  saturation  of  information,  meaning  that 
no  new  information  was  emerging  (Merriam,  1988).  Quotations,  comments,  and  observations 
were  written  down  on  field  notes  immediately  during  the  interviews.  Researchers  reviewed  the 
notes  after  the  session  and  made  expanding  comments  as  necessary. 

Data  Analysis 

Data  analysis  was  conducted  simultaneously  with  the  data  collection.  This  included  sorting  the 
responses  into  categories,  interpreting  the  data,  and  formatting  the  data  into  a  reflective  picture 
(Creswell,  1994).  As  the  data  were  collected,  coding  was  accomplished  to  organize  the 
information  into  themes  or  categories  for  interpretation.  Categories  for  the  codes  were 
developed  after  some  of  the  initial  interviews  were  completed  and  the  researchers  were  able  to 
cluster  responses  into  major  topics  and  themes.  Topical  codes  evolved  and  responses  were  re¬ 
coded  as  necessary  during  the  course  of  the  study  as  more  data  became  available  and  an 
understanding  of  the  topical  categories  was  refined.  Interview  axial  coding,  which  reflects  the 
interrelationships  among  the  responses,  is  detailed  in  Appendix  F  for  primary  data  collection  and 
Appendix  G  for  core  CRM  behavior  data  collection. 

Efforts  were  taken  during  data  collection  and  analysis  to  minimize  bias,  or  prejudicial 
outlook,  in  the  research  findings.  While  bias  can  be  introduced  into  a  qualitative  inquiry  process 
during  every  phase  of  the  research,  the  researchers  recognized  that  the  major  potential  source  of 
bias  lay  in  erroneously  interpreting  findings  to  be  consistent  with  the  researchers’  point  of  view 
(Potts,  1990).  Being  aware  of  this  pitfall,  two  researchers  were  always  used  to  take  separate 
notes. 

To  address  the  issue  of  reliability  between  researchers,  both  sets  of  notes  for  each  participant 
were  examined  side-by-side.  Each  researcher’s  written  notes  were  read  and  compared  to  those 
of  the  other  researcher  for  each  response.  While  some  differences  existed  in  style  and  nature  of 
the  recorded  responses  (that  is,  one  researcher  may  have  written  a  verbatim  response  while  the 
other  summarized  the  same  response),  content  of  the  responses  appeared  to  be  nearly  identical 
for  most  responses.  Where  content  differed,  it  was  not  contradictory,  suggesting  that  each 
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researcher  may  have  focused  more  specifically  on  different  aspects  of  long  responses.  Where 
these  differences  exist,  both  sets  of  responses  have  been  included  in  the  data. 

FINDINGS 

The  interviews  provided  a  rich  narrative  with  which  to  examine  and  understand  single-seat 
fighter  pilots’  attitudes  toward  CRM.  The  findings  were  initially  open  coded  under  the  topical 
subjects,  as  outlined  in  the  Interview  Worksheet  (Appendix  E):  CRM  formal  training  received; 
overall  comments  on  USAF  emphasis  on  CRM;  integration  of  CRM  into  everyday  flight  and 
simulator  operations;  instructors’  attitudes  of  students’  understanding  use  of  CRM;  students’ 
attitudes  of  instructors’  and  evaluators’  use  of  CRM;  interviewees’  suggestions  to  improve  CRM 
training;  and  assessment  of  learning  styles. 

As  interviews  were  conducted  and  on-going  analysis  performed,  new  response  groupings,  or 
topics,  evolved.  These  responses  were  axial  coded  into  more  meaningful  categories  and  became 
the  basis  for  data  reduction  and  findings.  The  axial  coding  for  formal  CRM  training  became: 
Neutral  or  mixed  experience,  strongly  positive  experience,  or  less  than  positive  experience.  The 
axial  coding  topics  for  attitudes  toward  F-16  CRM  training  program  became:  F-16  pilots  feel 
that  they  are  already  doing  CRM;  F-16  pilots  feel  that  they  are  being  “forced”  to  do  CRM 
training;  use  of  the  term  “CRM”  is  not  a  common  or  desired  point  of  reference  for  F-16  pilots;  F- 
16  training  should  focus  on  single-seat,  not  multicrew  aircraft;  USAF  needs  to  show  that  it  is 
serious  about  CRM;  other  positive  responses  and  other  negative  responses.  The  axial  coding 
topics  for  suggestions  by  respondents  to  improve  CRM  training  became:  Make  all  CRM  training 
tactically  relevant,  make  CRM  training  applicable  to  single-seat  fighter  pilots,  consider  using 
another  term  other  than  “CRM”  for  single-seat  fighters,  assure  course  delivery  applies  to 
audience,  assure  concept  of  training  (format,  duration,  frequency,  etc.)  enhances  learning 
process,  stress  that  leadership  should  support  training,  use  simulators  to  reinforce  training,  do  not 
force  training  to  be  accomplished,  assure'  CRM  instructors  have  credibility,  and  general 
comments. 

In  addition  to  tabulation  of  the  axial  coded  data,  representative  interview  comments  are 
included  to  add  depth  to  understanding  the  attitudes  of  the  F-16  pilots.  Axial  coding  of  the  data 
and  topic  selection  for  data  reduction  and  the  representative  comments  were  made  by  the 
principle  researcher,  a  former  F-16  pilot  and  fighter  wing  commander,  based  on  the  researcher’s 
twenty-five  years  of  interaction  with,  and  an  appreciation  for,  fighter  pilots  and  their  unique 
flight  environment. 


Interview  Topics 

The  first  findings  reported  are  pilots’  attitudes  concerning  CRM  formal  training  in 
undergraduate  pilot  training,  F-16  formal  course  training,  F-16  instructor  training,  F-16 
continuation  training,  and  other  formal  CRM  training.  The  next  findings  summarized  (Tables  1- 
7)  are  the  respondents’  attitudes  toward  the  USAF  emphasis  on  CRM,  integration  of  CRM  in 
everyday  operations,  instructors’  attitudes  of  students’  understanding  of  use  of  CRM,  students’ 
attitudes  of  instructors’  and  evaluators’  use  of  CRM,  suggestions  by  respondents  to  improve 
CRM  training,  and  respondents’  learning  styles.  Representative  responses  are  included  to 
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emphasize  the  depth  and  content  of  the  responses;  Appendix  F  is  a  complete  detailing  of  the 
primary  data  responses.  Additional  responses  on  associated  CRM  subjects  are  in  Appendix  G. 

CRM  Formal  Training 


Undergraduate  Pilot  Training  (UPT) 


Table  1.  UPT  Training 


Response 

Number 

Percentage 

Received  CRM-specific  training  in  UPT 

7 

20 

Did  not  receive  CRM-specific  training  in  UPT 

28 

80 

No  response 

1 

Representative  Respondent  Remarks. 

•  At  UPT  at  Sheppard  AFB  in  1993,  received  CRM  component  in  prioritization  and  task 
management,  but  they  did  not  know  how  to  deal  with  single-seat  CRM  concept  (8  to  12 
hours). 

•  Total  of  three  hours  of  classes:  At  beginning  of  UPT  as  overview,  halfway  through  T-37s  and 
at  end  of  T-38s.  Taught  by  physiology  training  people.  Mostly  general  information,  but  were 
shown  three  films  showing  crashes  with  good  and  bad  examples  of  CRM. 

•  In  T-37s/T-38s,  about  2  to  3  hours  of  lecture  on  risk  assessment,  CRM,  and  communication, 
using  mostly  videos  of  good  and  bad  CRM  and  mishaps. 

F-16  Formal  Training  (FT) 


Table  2.  F-16  Formal  Training 


Response 

Number 

Percentage 

18 

85.7 

No  training 

3 

14.3 

Table  3.  F-16  Formal  Training  Experience 


Response 

Number 

Percentage 

Neutral  or  mixed  experience 

10 

47.6 

Strongly  positive  experience 

0 

0.0 

Less  than  positive  experience 

8 

38.1 

No  training 

3 

14.3 
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Representative  Respondent  Remarks. 

•  “Classroom”  at  Luke,  4-hour  course,  in  generalized  physiological  training  and  situational 
awareness. 

•  Received  a  briefing  by  safety  people  that  was  supposed  to  be  on  how  CRM  applied  to 
fighters,  but  it  really  showed  how  “CRM  is  for  multicrew  airplanes.” 

•  Principles  in  bits  and  pieces  during  training,  but  “not  as  CRM  specific.” 

•  At  Mountain  Home  (1992),  initial  training  was  by  physiology  instructor.  Only  10%  useful 
and  couldn’t  remember  what  was  taught. 

F-16  Instructor  Course  (If 

Table  4.  F-16  Instructor  Course 


Response 

Number 

Percentage 

Some  training 

3 

30 

No  training 

7 

70 

No  response 

3 

Representative  Respondent  Remarks. 

•  In  academics  at  Luke  we  talked  a  little  about  CRM,  but  it  was  mostly  a  “square-filler.” 

•  Classroom  lecture  by  wing  aviation  physiologist  went  over  case  studies  for  about  two  hours. 

•  1  to  1  Vi  hours  at  Luke  on  common  errors  students  will  make. 

F-16  Continuation  Training  (CTt 
Total  CT  Training  fn  =  311 

Table  5.  F-16  Continuation  Training  Experience 


Experience 

Total 

(n) 

Total 

(%) 

Neutral  or  mixed 

18 

58.1 

Strongly  positive 

6 

19.3 

Less  than  positive 

7 

22.6 

The  representative  remarks  are  ordered  by  MAJCOM,  and  whether  the  comments  were  neutral, 
strongly  positive,  or  less  than  positive  experience. 
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AETC  fn  =  13) 


Table  6.  AETC  CT  Responses 


AETC 

AETC 

Experience 

(n) _ 

(%) 

Neutral  or  mixed 

12 

92.3 

Strongly  positive 

“  0 

0 

Less  than  positive 

1 

7.7 

Representative  Respondent  Remarks. 

Neutral  (or  mixed)  experience  (n=  12) 

•  CRM  training  in  operations  is  integrated  into  daily  flying  activities;  it  is  not  separated  as  a 
separate  subject. 

•  One  hour  of  discussion  using  slides  and  videos  on  incidents  and  accidents  taught  by  wing 
aviation  physiologist. 

•  1  to  1  Vi  hours  per  year  in  squadron  at  Luke,  given  by  the  Academic  Squadron  aviation 
physiologist,  covering  incidents  and  accidents  and  the  thought  process  that  occurred.  This  is 
a  “grounding  item”  if  it  is  not  attended. 

Less  than  positive  (n  =  1) 

One  hour  of  lecture  in  classroom:  “Everyone  hated  it.” 

ACC/USAFE/PACAF  fn  =  18) 

Table  7.  ACC/USAFE  CT  Responses 


ACC/ 

ACC/ 

Experience 

USAFE 

USAFE 

(n) 

(%) _ _ 

Neutral  or  mixed 

6 

33.3 

Strongly  positive 

6 

33.3 

Less  than  positive 

6 

33.3 

Representative  Respondent  Remarks. 

Neutral  (or  mixed)  experience  (n  =  6) 

•  At  Mountain  Home  in  1995  in  the  ACC  CRM  van.  First  couple  of  times  it  was  worthless 
because  it  had  a  heavy  focus  on  multiengine  CRM.  But  then  it  changed  to  move  to  more 
single-seat  focus  and  fighter  application  and  became  a  very  good  refresher.  PC  flight 
simulator  was  fun,  but  not  useful. 
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•  ACC  mobile  CRM  Van  at  Shaw  (twice).  Worthwhile  training,  but  it  was  a  little  long  because 
of  the  day-to-day  operational  tempo.  It  drives  things  like  shooting  the  gun  to  the  weekends. 

Strongly  positive  experience  (n  =  6) 

•  Tactical  scenarios  in  the  ACC  CRM  van  at  Mountain  Home  were  very  relevant. 

•  Six-hour  course  at  Spangdahlem  in  CRM  van  in  1996.  Case  studies  were  good,  but  CRM  or 
emergency  training  in  a  real  simulator  would  have  been  better  than  the  PC-based  simulators 
in  the  van. 

Less  than  positive  experience  (n  =  6) 

•  Initial  CT  training  at  Shaw  in  ACC  CRM  van  for  4  Vi  hours,  including  a  2-hour  hands-on 
trainer.  Some  of  the  exercises  were  very  useful  to  emphasize  CRM  issues.  However,  follow- 
on  CT  was  a  waste  of  time  for  single-seat  fighter  pilots  (except  for  F-15E  and  EF- 1 11s) 
because  it  emphasized  too  much  “crew  coordination.” 

•  Four  hours  at  Holloman  in  ACC  CRM  van  following  a  mission  profile  (1996).  “Not  very 
productive;  I  already  knew  that  I  was  supposed  to  use  all  of  my  resources.” 

Other  CRM  Formal  Training 

Representative  Respondent  Remarks. 

•  The  best  CRM  training  was  going  through  the  accident  investigation  school  at  the  Safety 
Center  at  Kirkland.  It  was  heavy  on  human  factors. 

•  As  a  former  navigator,  received  a  lot  of  crew  coordination  in  navigator  school,  Holloman 
Weapons  System  Officer  (WSO)  lead-in,  and  F-4  school. 

•  In  first  F-4  operational  unit  in  late  '80s,  received  CRM  that  was  applicable  only  to  heavy, 
multicrew  aircraft. 

•  Some  CRM  at  centrifuge  training  on  task  management  and  psychological  training  (1  ‘A 
hours). 


Attitudes  toward  F-16  CRM  Training  Program 

The  F-16  pilots’  responses  to  the  interview  questions  exploring  their  attitudes  toward  CRM 
training  were  recoded  to  reflect  common  topic  areas  (see  Table  8).  Representative  remarks 
follow  to  add  narrative  depth  to  the  topic  codings.  For  details  of  the  complete  responses,  refer  to 
Appendix  F. 
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Table  8.  Axial  Coding  of  Interview  Responses 


Interview  Topics 

Coding 

Area 

USAF 

emphasis 

Integration 
top  down 
guidance 

Integration 

flight 

operations 

Integration 

Simulator 

Operations 

IP 

attitudes 

Student 

attitudes 

Total 

responses 

Already 

doing 

CRM 

11 

4 

2 

- 

i 

- 

18 

Being  forced 
to  do  CRM 
training 

6 

7 

1 

- 

- 

1 

15 

CRM  not 
desired  term 

9 

16 

19 

7 

10 

5 

66 

Need  to  focus 
on  fighters 

4 

1 

2 

. 

7 

USAF 
must  show 
it’s  serious 

5 

1 

- 

- 

- 

- 

6 

Other 

positive 

comments 

9 

6 

13 

18 

5 

2 

53 

Other 

negative 

comments 

6 

6 

8 

15 

6 

- 

41 

Representative  Respondent  Remarks 

F-16  pilots  feel  that  they  are  already  doing  CRM. 

•  There  is  a  lot  of  resistance  to  CRM  training  because  the  fighter  community  has  been  doing 
this  for  years;  it  is  just  that  they  did  not  call  it  anything. 

•  We  know  about  CRM  “once  in  a  while  by  hearing  a  buzzword,  but  it’s  really  a  different 
name  for  something  we’ve  already  been  doing.” 

•  Fighter  pilots  already  do  CRM,  and  it’s  “common  sense.” 

•  “CRM  has  always  been  done  in  the  single-seat  world,  that’s  why  single-seat  guys  ‘cut-up’  on 
CRM.” 

F-16  pilots  feel  that  they  are  being  “forced”  to  do  CRM; 

•  “CRM  training  is  something  that  you’re  forced  to  do  to  fill  a  square.” 

•  Very  negative  opinion.  “It’s  just  another  one  of  those  things  we  have  to  do;  the  latest 
management  de  jour.” 

•  Principles  should  be  applied,  but  no  one  wants  another  line  in  the  briefing  guide  that  says: 
“CRM.” 
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Use  of  the  term  “CRM”  is  not  a  common  or  desired  point  of  reference  for  F-16  pilots. 


It  is  being  implemented  and  covered,  but  it’s  not  called  CRM  because  “CRM  is  for  multi¬ 
crew  airplanes.” 

CRM  is  something  that  is  aligned  with  a  crew  concept  and  is  not  applicable  to  single-seat 
fighter  pilots,  but  “CRM  concepts”  are  ok  and  applicable. 

“Never  refer  to  CRM  as  ‘CRM’  because  it  will  turn  people  off!” 

Word  CRM  creates  “hostile”  attitudes. 

Students  understand  the  subtasks  of  CRM  but  do  not  like  the  term  “Crew  Resource 
Management”  because  they  say  that  they  are  “not  in  a  crew  airplane.” 

Single-seat  pilots  hate  words  with  “crew”  in  them.  They  want  to  use  pilot  instead  of  crew, 
for  example:  pilot  van  vice  crew  van;  pilot  rest  vice  crew  rest;  and  pilot  meeting  vice  crew 
meeting. 

F-16  CRM  training  should  focus  on  single-seat,  not  multicrew,  aircraft. 

“CRM  is  not  really  being  addressed  toward  a  single-seat  audience.” 

Effort  is  good,  but  CRM  needs  to  be  specialized  for  single-seat  aircraft  using  flight 
communication  techniques,  instead  of  using  multiplace  airplanes  and  flight  deck 
communication  procedures. 

“Most  CRM  training  has  been  focused  on  crew-planes,  so  it’s  not  always  applicable,  but  the 
single-seat  guys  try  to  translate  it.” 


USAF  needs  to  show  that  it  is  serious  about  CRM. 

Great  reluctance  of  the  captain-on-the-street  to  adopt  something  unless  they  see  that  the  Air 
Force  is  serious  about  implementing  it.” 

Not  sure  people  know  what  CRM  is.  Some  like  it;  some  think  it  is  bunk.  There  is  a  lack  of 
dollars  to  support  the  CRM  program,  but  “we  talk  a  good  game.” 

The  Air  Force  has  placed  emphasis  on  CRM,  but  it  is  not  clear  in  the  wing  or  squadron  as  to 
the  specific  direction. 

Other  positive  comments. 

While  people  have  not  been  responsive  to  CRM  in  the  past,  they  seem  to  be  getting  better. 
“Formal  program  is  a  step  in  the  right  direction.” 

CRM  training  is  useful  if  it  is  done  the  right  way  and  not  just  filling  a  square.  It  should  be 
more  frequent  and  not  just  annually. 

Mentoring  is  very  important  for  the  IPs  to  show  students  the  importance  of  CRM  concepts. 
Other  negative  comments. 

Every  two  or  three  years,  the  Air  Force  comes  up  with  a  new  “program  to  worship.” 

You  should  manage  your  cockpit  yourself;  you  don’t  need  help. 


•  CRM  is  not  integrated  into  simulator  missions,  and  they  should  be. 

•  “Students  come  into  F-16  formal  training  too  CRM  oriented  and  are  too  willing  to  speak  up 
right  away.  They  forget  that  they  are  in  a  single-seat  fighter.” 

Suggestions  bv  Respondents  to  Improve  CRM  Training 

The  F-16  pilots’  suggestions  to  improve  CRM  training  have  been  recoded  to  reflect  common 
topic  areas  (see  Table  9).  The  responses  in  Table  9  indicate  the  frequency  of  the  suggestions. 
Representative  remarks  follow  to  add  narrative  depth  to  the  topic  coding.  For  details  on  the  full 
responses,  refer  to  Appendix  F. 

Table  9.  Suggestion  Topics  to  Improve  CRM  Training. 


Suggestion 

Responses 

Assure  concept  of  training  (format,  duration,  frequency,  etc.)  enhances 
learning  process 

30 

Assure  course  delivery  applies  to  audience 

22 

Consider  using  another  term  other  than  “CRM”  for  single-  seat  fighters 

20 

Make  all  CRM  training  tactically  relevant 

13 

General  comments 

13 

Make  CRM  training  applicable  to  single-seat  fighters 

9 

Use  simulators  to  reinforce  training 

7 

Do  not  force  training  to  be  accomplished 

6 

5 

3 

Representative  Respondent  Remarks 

Make  all  CRM  training  tactically  relevant. 

•  Use  tactical  scenarios  in  training  to  make  it  interesting  and  keep  students’  minds  thinking 
about  it. 

•  Minimize  psychological  terms. 

•  The  more  the  tactical  relevance  of  the  CRM  principles  can  be  applied,  the  better  the  outcome 
of  the  mission. 

•  Anytime  you  work  with  fighter  pilots,  you  will  have  challenges.  Must  stress  “practicality” 
and  “relationship  to  mission  objectives.”  No  fluff! 

Make  CRM  training  applicable  to  single-seat  fighters. 

•  Make  it  applicable  to  single-seat  fighter  pilots. 

•  Think  of  CRM  from  the  perspective  of  a  single-seat  pilot. 
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•  The  effort  is  good,  but  CRM  needs  to  be  specialized  for  single-seat  aircraft  using  flight 
communication  techniques,  instead  of  using  multiplace  airplane  flight  deck  communication 
procedures. 

Consider,  using  another^  term  for  single-seat  fighters  other  than  Cockpit/Crew  Resource 

Management. 

•  “CRM  is  a  dirty  word  to  the  fighter  community,”  don’t  ever  refer  to  CRM,  but  just  teach  the 
subtopics. 

•  Change  the  name  CRM  to  something  that  is  not  labeled  as  being  from  the  heavy  community. 

•  De-emphasize  multicrew  CRM  (the  genesis  of  CRM).  The  reaction  of  the  fighter  world  to 
CRM  is  to  “scoff’  at  it. 

•  F-16  pilots  “shut  off’  at  the  use  of  the  term  “CRM.”  It  is  not  for  F-16  pilots  since  they  have 
no  crews. 

•  It  matters  what  you  call  the  training.  “Do  not  call  it  CRM.” 

Assure  course  delivery  applies  to  audience. 

•  Use  of  case  studies  of  accidents  is  very  good. 

•  Do  not  use  old  videos  of  accidents  and  crashes  over  and  over  to  motivate. 

•  Have  more  “hands-on”  training. 

•  Do  not  use  big-room  lectures  with  a  lot  of  “concepts”. 

Assure  concept  of_  training  (format,  duration,  frequency,  etc.)  enhances  learning 

process. 

•  Have  a  mixture  of  experience  in  the  class  to  keep  it  interesting. 

•  If  CRM  training  takes  too  long  of  a  time,  single-seat  fighter  pilots  will  go  to  it  with  a 
negative  attitude.  “The  shorter  you  can  make  continuation  training  (one  hour  desired),  the 
better.” 

•  CRM  training  is  useful  if  it  is  done  the  right  way  and  not  just  filling  a  square.  It  should  be 
more  frequent  and  not  just  annually. 

•  Must  stress  that  CRM  is  for  everything,  not  just  emergencies  and  contingencies. 

Leaders  must  support  t raining . 

•  Squadron  Commanders,  Operations  Officers,  and  Weapons  Officers  must  embrace  it. 

•  CRM  issues  should  be  covered  often  in  briefings. 

•  Present  it  as  part  of  normal  pilots’  meetings. 

Use  simulators  to  reinforce  training. 

•  Use  more  interactive  training  and  less  classroom  time.  Need  a  PC-based,  nonclassified, 
communications  interactive,  F-16  simulator  for  pilots  to  practice  mission  at  home. 
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•  Use  video  playback  of  simulator  or  training  device  missions  that  include  the 
“communication”  between  the  players. 

•  Have  simulator  missions  specially  designed  for  CRM  with  multiple  roles  played  by  the 
simulator  instructor,  off  of  a  script,  stressing  interflight  and  external  communications  (SOF, 
etc). 

Do  not  “force”  training  to  be  accomplished. 

•  “We  already  have  enough  ground  duties,  don’t  increase  them.” 

•  We  do  CRM  already;  we  don’t  need  to  be  trained  to  do  it. 

•  Do  not  change  things  and  increase  the  workload  of  the  instructors  by  forcing  them  to 
formally  do  something  that  they  have  always  done. 

CRM  instructors  must  have  credibility. 


•  Must  keep  credible  (fighter  experienced)  CRM  instructors. 

•  Delivery  of  CRM  training  is  very  important;  audience  must  relate  to  the  instructor  as  far  as 
background  and  knowledge. 

•  Do  not  have  the  aerospace  physiologist  teach  CRM  because  they  do  not  have  the  credibility 
to  F-16  pilots.  Have  F-16  pilots  teach  F-16  pilots. 

General  comments. 


•  Keep  training  applicable.  Use  a  format  that  “takes  you  out  of  the  squadron  and  allows  you  to 
focus.” 

•  CRM  seems  “detached  from  the  rest  of  the  training  syllabuses;  it  should  be  integrated." 

•  Consider  ops  doing  some  CRM  with  maintenance  personnel. 

Assessment  of  Learning  Styles 

The  way  people  learn  best,  or  their  learning  style,  is  of  considerable  significance  in 
developing  and  delivering  successful  training  programs.  One  model  suggests  that  there  are  three 
recognized  primary,  or  dominant,  learning  styles:  First,  visual  learners,  who  learn  best  by 
reading  or  looking  at  pictures.  Second,  auditory,  or  aural,  learners,  who  learn  best  by  listening. 
And  third,  hands-on,  tactile,  or  kinesthetic  learners,  who  need  to  use  their  hands  or  whole  body 
to  learn  (Filipczak,  1995).  In  developing  CRM  training  programs,  it  is  important  to  understand 
how  people  learn  the  best  for  long-term  retention  and  future  application.  Because  of  the  depth 
and  complexity  of  the  subject  matter,  aviation  instructors  must  present  the  course  material  in 
ways  that  satisfy  the  different  needs  and  styles  of  aviation  learners.  This  study  assessed  the 
learning  styles  of  the  F-16  pilots  and  found  over  half  were  hands-on  learners,  and  the  rest  were 
either  visual  learners  or  a  combination  of  hands-on  and  visual  learners  (see  Table  10).  The 
composition  of  this  learning  style  breakout  mirrors  previous  research  on  aviators  (Karp,  1998). 
Of  particular  note  is  that  there  were  no  audio  only  learners,  which  is  the  predominant  form  of 
most  lectures. 


15 


Table  10.  F-16  Pilot  Learning  Styles. 


Learning  Style 

Respondents  (n  =  36) 

Percentage 

Hands-on  Learner 

17 

48.6 

Visual  Learner 

9 

25.7 

Hands-orWisual  Learner 

6 

17.1 

Visual/Auditory  Learner 

3 

8.6 

Auditory  Learner 

0 

0  1 

Hands-on/Auditory  Learner 

0 

0 

No  Assessment  Made 

1 

2.7 

Representative  Respondent  Remarks 
Visual  Learners  (n  =  9,  25.7%). 

•  Visual,  followed  up  by  flying. 

•  Must  see  it,  instead  of  hear  it. 

•  If  I  can  see  a  good  demo  or  a  picture,  I  can  recreate  it  later. 

Hands-on  Learners  (n  =  17,  48.6%L 

•  I  have  to  read  a  little,  but  I  need  to  be  hands-on  to  really  learn. 

•  I  am  slow  to  learn  right  away,  but  when  I  touch  something,  my  learning  accelerates. 

•  I  learn  by  “doing  it.” 

•  Must  write  or  do  the  task  to  learn. 

Hands-on/Visual  Learners  (n  =  6, 17.1%L 

•  Hands-on  for  flying,  visual  for  systems. 

•  Visual  learner  with  hands-on  a  close  second. 

Visual/Auditory  (n  =  3.  8.6%L 

•  Must  do  both. 

No  Assessment  Made  (n  =  11. 


Additional  Findings 

While  reviewing  participant  responses  to  open-ended  interview  questions,  several  high 
response  trends  were  identified  in  the  data.  Frequency  statistics  were  run  on  these  trends  to 
determine  the  valid  percent  of  participants  indicating  these  common  responses  (see  Table  11).  It 
is  important  to  note  that  these  trends  were  not  specifically  asked  questions,  but  they  occurred 
spontaneously  during  interviews  relating  to  CRM  core  behaviors  and  skills.  These  inferences 
could  be  different  if  these  questions  were  asked  directly  (See  Appendices  F  and  G). 
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Table  1 1 .  Common  Primary  Data  and  Core  CRM  Behavior  Data  Responses 


Common  participant  response 

Valid  percent  of 
participants  voicing 
this  response 

Indicated  that  fighter  pilots  already  use  CRM  principles 

91 

Indicated  that  CRM  is  applicable  to  fighters 

94 

Indicated  that  they  do  not  like  the  term  CRM 

40 

Indicated  that  CRM  is  done  but  is  not  called  CRM  in  fighters 

82 

Indicated  that  CRM  principles  are  important  in  briefing 

97 

Indicated  that  CRM  principles  are  important  in  debriefing 

87 

Indicated  that  a  good  flight  lead  is  characterized  by: 

-Being  a  good  communicator 

58 

-Having  good  people  skills 

39 

-Being  knowledgeable 

33 

-Being  a  good  pilot 

15 

-Aggressiveness 

15 

-Being  a  good  overall  leader 

6 

Indicated  that  wingmen  should  speak  up: 

-In  flight 

91 

-When  issue  may  affect  mission  (including  training  mission) 

47 

-During  briefing  &  debriefing 

41 

-For  bad  feelings  or  if  uncomfortable  with  situation 

33 

Preferred  method  for  wingmen  to  question  or  non-concur  with 

lead  during  flight 

-Plain  English 

40 

-Knock  it  off  call 

30 

-Ask  questions 

23 

-Other  methods 

7 

INTERPRETATIONS,  CONCLUSIONS,  IMPLICATIONS,  AND 
RECOMMENDATIONS 

The  research  interpretations,  conclusions,  and  implications  were  determined  by  the  principle 
researcher,  a  former  F-16  pilot  and  fighter  wing  commander,  based  on  the  researcher’s  twenty- 
five  years  of  interaction  with,  and  appreciation  of,  fighter  pilots  and  their  unique  flight 
environment.  In  qualitative  research,  the  success  of  interviews  rests  on  the  skill  and  experience 
of  the  investigators  (Merriam,  1988).  Development  of  these  research  interpretations, 
conclusions,  and  implications  was  derived  from  not  only  exactly  “what”  each  respondent  said 
and  “how”  they  said  it,  but  also  from  what  the  researchers  assessed  was  their  underlying 
“attitudes”  toward  the  research  questions. 
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Interpretations  (from  data) 
CRM  Training 


Undergraduate  Pilot  Training  (UPT).  Only  7  of  the  35  respondents  indicated  that  they  were 
given  CRM  training  during  UPT.  A  high  correlation  between  rank  (an  approximation  of  time  in 
the  Air  Force)  and  the  respondents  who  stated  that  they  had  not  received  specific  CRM  training 
in  UPT,  reflects  the  recent  introduction  of  CRM  training  over  the  few  years  (see  Table  12). 

Table  12.  UPT  Training 


Response 

Number 

Percentage 

Received  CRM-specific  training  in  UPT 

7 

20 

Did  not  receive  CRM-specific  training  in  UPT 

28 

80 

No  response 

1 

Potential  correlation  between  demographic  variables  and  common  response  variables  were 
examined.  Only  one  significant  relationship  was  found:  the  relationship  of  rank  with  completion 
of  a  CRM  training  course  during  UPT  (r  =  .632,  p  <  .01).  With  relatively  new  pilots  being  given 
training  in  UPT,  which  the  older  pilots  have  not  been  exposed  to,  a  common  framework  of 
CRM-related  terms  and  expectations  between  those  groups  appears  to  be  still  evolving.  This 
variance  in  foundational  CRM  formal  training  in  UPT  could  contribute  to  some  differences  in 
attitudes  toward  CRM  concepts  and  principles  between  the  older  pilots  and  the  younger  pilots. 
The  data  also  revealed  that  formal  CRM  training  in  UPT  were  not  consistent  among  the 
respondents  with  respect  to  either  syllabus  structure  or  content.  However,  newly  published 
AETC  Instructor  and  Student  CRM  lesson  plans  underscore  a  move  by  AETC  to  standardize 
baseline  CRM  training  in  UPT  and/or  Undergraduate  Navigator  Training  (UNT). 

F-16  Formal  CRM  Training.  F-16  Formal  Training  (FT)  CRM  program  mirrors  the  same 
gradual  CRM  knowledge  evolvement  experienced  by  UPT  pilots,  with  the  youngest  pilots 
having  received  the  most  CRM  exposure  (see  Tables  13  and  14). 

Table  13.  F-16  Formal  Training 


Response 

Number 

Percentage 

Some  training 

18 

85.7 

No  training 

3 

14.3 

Table  14.  F-16  Formal  Training  Experience 


Response 

Number 

Percentage 

Neutral  or  mixed  experience 

10 

47.6 

Strongly  positive  experience 

0 

0.0 

Less  than  positive  experience 

8 

38.1 

No  training 

None 

14.3 

18 


Until  recently,  expanded  AETC  CRM  lesson  plans  did  not  exist  in  a  format  that  completely 
aligned  with  the  Air  Force  Instruction  on  CRM  training  programs,  the  latest  being  AFI  11-290, 
dated  14  August  1998.  Consequently,  without  standardized  direction,  the  FT  training  programs 
appeared  to  be  oriented  toward  aviation  physiology  subjects,  such  as  G-LOC  and  stress 
management,  as  well  as  some  CRM  subjects  like  problem  solving,  situational  awareness,  and 
aircraft  incident  and  accident  reviews.  Using  the  new  AETC  Instructor  and  Student  Guides, 
P-V4A-A-C-CR-IG,  March  1998,  as  a  baseline  upon  which  to  build,  FT  CRM  training  can  be 
expanded. 

F-16  Instructor  CRM  Training.  While  instructor  CRM  training  must  cover  a  broad  spectrum 
of  academic  and  flying  skills,  it  does  not  appear  to  have  a  consistent  component  which  focuses 
on  teaching  instructors  to  recognize  or  correct  shortfalls  in  the  students’  understanding  of,  or 
implementation  of,  CRM  behaviors  and  skills  (see  Table  15). 


Table  15.  F-16  Instructor  Course 


Response 

Number 

3 

No  training 

7 

1 

No  response 

3 

If  the  instructors  are  to  carry  the  responsibility  of  assuring  that  CRM  skills  and  behaviors  are 
a  part  of  every  flight  and  simulator,  then  the  instructors  should  have  continual  CRM  refresher 
training  to  highlight  recent  innovations  and  developments  in  CRM  and  aviation  human  factors. 

F-16  Continuation  Training.  Major  commands  individually  administer  CRM  CT  training 
programs.  AETC  and  PACAF  support  CT  with  in-house  resources,  while  ACC  and  USAFE 
provide  this  support  through  a  contractor.  The  F-16  pilots’  experiences  with  the  commands’ 
training  were  varied  (see  Table  16). 


Table  16.  Continuation  Training 


Experience 

AETC 

(n) 

AETC 

(%) 

ACC/ 

USAFE 

(n) 

ACC/ 

USAFE 

(%) 

Total 

(n) 

Total 

% 

Neutral  or  mixed 

12 

92.3 

6 

33. 

18 

58.1 

Strongly  positive 

0 

0 

6 

6 

19.3 

Less  than  positive 

1 

7.7 

6 

33.3 

7 

22.6 

Many  F-16  pilots  who  attended  the  ACC  and  USAFE  CRM  training  programs  liked  the  case 
studies  with  interactive  computer-based  training,  but  did  not  like  the  PC-based  situational  trainer. 
The  tactical  relevance  of  the  training  and  applicable  scenarios  were  very  high  on  the 
respondents’  priorities.  Experience  and  credibility  of  CRM  instructors  were  also  very  important. 
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Other  CRM  Training.  CRM  training  has  been  conducted  in  various  formats  by  organizations 
across  the  Air  Force;  however,  prior  to  the  issuance  of  AFI  1 1-290  and  its  predecessor  AFI  36- 
2243,  there  had  not  been  an  effort  to  develop  an  overarching  CRM  training  program. 

Summary  Interpretation  of  CRM  Training.  Currently,  each  major  command  is  responsible 
for  developing  its  own  CRM  program.  Consequently,  there  have  been  variances  in  content  and 
magnitude  of  CRM  training  that  pilots  receive  in  UPT,  formal  training,  and  continuation  training 
in  the  operational  units.  These  training  programs,  which  may  differ  in  delivery  format  and 
subject  matter,  present  an  environment  in  which  pilots  may  understand  CRM  skills  and  behaviors 
from  different  perspectives  and  to  different  retention  and  application  levels.  Moving  between 
major  commands  could  also  result  in  negative  learning.  Additionally,  pilots  who  have  recently 
gone  through  UPT  and  initial  weapons  systems  training  have  more  formal  CRM  training  than 
pilots  who  have  been  in  the  Air  Force  longer.  This  could  create  a  situation  where  the  “older” 
pilots,  including  instructors  and  flight  leaders,  do  not  view  CRM  behaviors  and  skills  with  the 
same  perspective  as  the  “younger”  pilots. 

Attitudes  Toward  F-16  CRM  Training  Program 

Pilots’  “perceptions”  about  CRM  appeared  in  some  cases  to  be  an  obstacle  to  adoption  of  CRM 
principles  in  the  broadest  context  (see  Table  17).  The  F-16  pilots  interviewed  displayed  a  high 
enthusiasm  for  their  mission  and  an  in-depth  knowledge  of  fighter  operations.  However,  many 
pilots  did  not  indicate  a  full  understanding  of  the  breadth  of  the  Air  Force  CRM  program,  or  the 
value  of  CRM  training  to  enhanced  combat  readiness  and  effectiveness.  Many  experienced 
single-seat  F-16  fighter  pilots  felt  that  they  had  “always  done  CRM.”  They  generally  did  not  like 
the  term  “CRM”  because  of  multicrew  airplane  connotations.  Many  pilots  thought  CRM  focused 
on  multicrew  aircraft.  Some  pilots  indicated  they  wanted  the  USAF  to  show  it  was  serious  about 
CRM  training  by  funding  it  adequately  and  assuring  that  CRM  training  was  specialized  for 
single-seat  fighter  aircraft.  Potential  modifications  to  single-seat  fighter  CRM  should  consider 
the  sensitivities  and  attitudes  of  the  pilots  to  enhance  CRM  acceptance  and  incorporation  into 
everyday  flight  operations. 


Suggestions  to  Improve  CRM  Training 

The  three  highest  frequencies  of  F-16  pilots’  suggestions  to  improve  CM  training  were  to:  assure 
the  concept  of  training  (format,  duration,  frequency,  etc.)  enhances  the  learning  process,  assure 
course  delivery  is  applicable  to  the  audience,  and  consider  using  another  term  other  than  “CRM” 
for  single-seat  fighters.  The  next  three  highest  responses  were  to:  make  sure  CRM  training  was 
tactically  relevant,  make  CRM  training  applicable  to  single-seat  fighters,  and  use  simulators  to 
reinforce  CRM  training  (see  Table  1 8). 

Based  on  the  strength  of  the  pilots’  responses,  these  priorities  should  form  a  basis  for 
considering  alterations  to  the  CRM  training  program  to  facilitate  single-seat  fighter  pilots 
participation  in  CRM  training  process. 
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Table  17.  Axial  Coding  of  Responses 


Interview  Topics 

Coding 

Area 

USAF 

emphasis 

Integration 
top  down 
Guidance 

Integration 

flight 

operations 

Integration 

simulator 

operations 

IP 

attitudes 

Student 

attitudes 

Total 

responses 

Already 
doing  CRM 

11 

4 

2 

1 

_ 

18 

Being  forced 
to  do  CRM 
training 

6 

7 

1 

- 

- 

i 

15 

CRM  not 
desired  term 

9 

16 

19 

7 

10 

5 

66 

Need  to  focus 
on  fighters 

4 

1 

2 

7 

USAF  must 
show  it’s 
serious 

5 

1 

- 

- 

- 

6 

Other 

positive 

comments 

9 

6 

13 

18 

5 

2 

53 

Other 

negative 

comments 

6 

6 

8 

15 

6 

“ 

41 

Table  18.  Respondent  Suggestion  Topics  to  Improve  CRM  Training. 


Suggestion 

Responses 

Assure  concept  of  training  (format,  duration,  frequency,  etc)  enhances 
learning  process 

30 

Assure  course  delivery  applies  to  audience 

22 

Consider  using  another  term  other  than  “CRM”  for  single-  seat  fighters 

20 

Make  all  CRM  training  tactically  relevant 

13 

General  comments 

13 

Make  CRM  training  applicable  to  single-seat  fighters 

9 

Use  simulators  to  reinforce  training 

7 

Do  not  force  training  to  be  accomplished 

6 

Leadership  must  support  training 

5 

CRM  instructors  must  have  credibility 

3 

Interpretation  of  Assessment  of  Learning  Styles 

The  F-16  pilots  interviewed  self-identified  themselves  as  48.6%  hands-on  learners  and 
17.1%  a  combination  of  hands-on  and  visual  learning  (see  Table  19).  CRM  courses  should  be 
designed  to  present  material  in  more  than  one  learning  style  to  enhance  learning. 


21 


Table  19.  F-16  Pilot  Learning  Styles. 


Learning  Style 

Respondents  (n  =  36) 

Percentage 

Hands-on  Learner 

17 

48.6 

Visual  Learner 

9 

25.7 

Hands-on/Visual  Learner 

6 

17.1 

Visual/Auditory  Learner 

3 

8.6 

Auditory  Learner 

0 

0 

Hands-on/ Auditory  Learner 

0 

0 

No  Assessment  Made 

1 

2.7 

Because  research  also  indicates  that  aviators  tend  to  be  predominantly  hands-on  learners, 
immediate  hands-on  application,  such  as  using  distributed  mission  training  (DMT),  simulators, 
interactive  computer-based  training  or  interactive  training  devices,  is  very  important.  Research 
in  advanced  learned  models  for  aviation  education  also  reveals  that  in  addition  to  incorporation 
of  all  learning  styles  for  knowledge  transfer,  including  the  use  of  immediate  hands-on  application 
to  improve  long-term  retention  and  application,  the  adoption  of  adult,  cooperative,  and 
observational  learning  principles  and  techniques  should  have  reinforcing  value  in  developing 
CRM  training  programs  (Karp,  1998). 

Over  the  last  few  decades,  the  learning  model  that  has  been  generally  used  in  all  aviation 
education  has  remained  predominantly  unchanged.  Whether  it  involves  the  flight  members,  crew 
members  in  the  front  of  the  aircraft,  or  the  mission  crews  in  the  back,  aviation  education  has 
historically  involved  a  highly  structured  presentation  of  information  in  a  lecture  format,  possibly 
followed  sometime  in  the  fbture  by  practice  in  a  simulator  or  the  aircraft.  While  this  “lecture 
now,  application  in  the  future”  model  of  knowledge  transfer  may  work  relatively  well  with 
younger  learners,  research  has  shown  that  aviators,  as  well  as  other  learners  involved  in  highly 
technical  courses  of  study,  respond  more  efficiently  to  an  adult  learning  model  (Karp,  1996). 
When  adult  learning  is  combined  with  cooperative  learning,  learning  style  theory,  and  immediate 
application,  the  resulting  integrated  CRM  learning  model  can  become  a  powerful  tool  to  transfer 
CRM  knowledge  for  long-term  retention  and  enhanced  application  (see  Figure  1). 

Integrated  CRM  Learning  Model 

Research  confirms  that  mature  adults  learn  differently  than  younger  learners.  In  fact,  adults  learn 
best  when  they  believe  that  they  have  a  need  to  learn  and  are  ready  to  learn.  Normally,  adults 
are  self-directed  learners  and  require  an  instructor  to  paint  a  clear  picture  of  where  a  course  is 
going,  and  why,  before  they  are  willing  to  commit  themselves  to  a  learning  enterprise.  Adult 
learners  are  unwilling  to  wait  until  some  time  in  the  future  to  use  their  knowledge;  they  want  to 
apply  knowledge  immediately.  Furthermore,  research  has  shown  that  while  adults  are  least  likely 
to  learn  in  a  “lecture-only”  environment,  they  learn  exceptionally  well  in  guided  discussions, 
after  they  have  been  exposed  to  the  baseline  knowledge.  Additionally,  adult  learners,  such  as 
aircrews  who  use  highly  sophisticated  technology,  respond  very  positively  to  multimedia 
presentations  when  they  are  integral  to  those  facilitated  discussions.  The  use  of  visually  and 
auditorially  engaging  computer-based  training,  which  could  include  DMT,  simulators,  and  other 
interactive  training  devices,  can  enable  aircrews  to  immerse  themselves  in  the  application  of 
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Figure  1 .  Integrated  Cockpit/Crew  Resource  Management  Learning  Model 


CRM  skills  during  the  multimedia  learning  process.  Also,  videotape  capture  of  key  events  in 
DMT,  simulator,  and  training  device  missions  during  unfolding,  complex  CRM  scenarios, 
coupled  with  facilitated  discussions,  can  be  highly  valuable  reinforcement  training  vehicles. 
Adults  also  learn  very  well  from  each  other.  Again,  research  indicates  that  by  being  involved  in 
discussions  within  a  peer  group,  adults  are  very  successful  in  attaining  knowledge  and  retaining 
it.  Additionally,  adults  benefit  from  observing  others,  especially  peers  who  have  attained  the 
group’s  respect,  like  flight  instructors  or  evaluators.  Adult  learning  is  also  enhanced  by  working 
in  groups,  especially  if  they  are  involved  in  problem-centered  discussions.  Working  a  problem 
with  multiple  facets  is  a  more  effective  tool  for  reinforcing  a  long-term  application  rather  than 
memorizing  a  series  of  definitions  and  facts  (Karp,  1998). 

Additional  CRM  Research  Topics 

While  reviewing  participant  responses  to  open-ended  interview  questions,  several  high 
response  trends  were  identified.  It  is  important  to  note  that  these  trends  were  not  specifically 
asked  questions,  but  they  occurred  spontaneously  during  interviews  relating  to  CRM  core 
behaviors  and  skills  (see  Tables  20  and  21  and  Appendices  F  and  G). 

The  open  coding  of  research  data  detailed  in  Appendix  G  offers  a  rich  narrative  for 
examining  F-16  pilots’  specific  attitudes  toward  CRM  behaviors  and  skills.  When  axial  coded, 
this  data  can  provide  a  picture  of  the  interrelationships  between  the  topical  subject  areas.  An 
expansion  of  this  research  could  be  accomplished  by  using  expanded  observational  research  a 
broad  segment  of  both  single-seat  fighters  and  multicrew  fighters. 

Summary  of  Interview  Response  Interpretations 

The  F-16  pilots  interviewed  displayed  a  high  enthusiasm  for  their  mission  and  an  in-depth 
knowledge  of  fighter  operations.  However,  many  pilots  did  not  indicate  a  full  understanding  of 
the  breath  of  the  Air  Force  CRM  program,  or  the  transfer  of  the  training  to  enhance  combat 
readiness  and  effectiveness. 

Currently  each  major  command  is  responsible  for  developing  its  own  CRM  program. 
Consequently,  there  have  been  variances  in  the  content  and  magnitude  of  CRM  training  that 
pilots  have  received  in  UPT,  formal  training,  and  continuation  training  in  the  operational  units. 
These  training  programs,  which  may  differ  in  delivery  format  and  subject  matter,  could  present 
an  environment  in  which  pilots  may  understand  CRM  skills  and  behaviors  from  different 
perspectives  and  to  different  retention  and  application  levels.  Transferring  between  major 
commands,  and  their  different  CRM  training  programs,  could  result  in  negative  learning. 
Additionally,  pilots  who  have  recently  gone  through  UPT  and  initial  weapons  systems  training 
have  more  formal  CRM  training  than  pilots  who  have  been  in  the  Air  Force  longer.  This  could 
create  a  situation  where  the  “older”  pilots,  including  instructors  and  flight  leaders,  do  not  view 
CRM  behaviors  and  skills  with  the  same  perspective  as  the  “younger”  pilots. 
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Table  20.  Common  Primary  Data  Responses 


Common  participant  response 

Percent  of 
participants 

Indicated  that  fighter  pilots  already  use  CRM  principles 

91 

Indicated  that  CRM  is  applicable  to  fighters 

94 

Indicated  that  they  do  not  like  the  term  CRM 

40 

Indicated  that  CRM  is  done  but  is  not  called  CRM  in  fighters 

82 

Table  21 .  Common  Core  CRM  Behavior  Data  Responses 


Common  participant  response 

Valid  percent  of 
participants  voicing 
this  response 

Indicated  that  CRM  principles  are  important  in  briefing 

97 

Indicated  that  CRM  principles  are  important  in  debriefing 

87 

Indicated  that  a  good  flight  lead  is  characterized  by: 

-Being  a  good  communicator 

58 

-Having  good  people  skills 

39 

-Being  knowledgeable 

33 

-Being  a  good  pilot 

15 

-Aggressiveness 

15 

-Being  a  good  overall  leader 

6 

Indicated  that  wingmen  should  speak  up: 

-In  flight 

91 

-When  issue  may  affect  mission  (including  training  mission) 

47 

-During  briefing  &  debriefing 

41 

-For  bad  feelings  or  if  uncomfortable  with  situation 

33 

Preferred  method  for  wingmen  to  question  or  non-concur  with 
lead  during  flight 

-Plain  English 

40 

-Knock  it  off  call 

30 

-Ask  questions 

23 

-Other  methods 

7 

Conclusions 

(From  interview  response  data) 

1.  Potential  modifications  to  single-seat  fighter  CRM  should  incorporate  the  pilots’ 
perceptions,  sensitivities,  and  attitudes  to  enhance  CRM  acceptance  and  incorporation  into 
everyday  flight  operations. 

The  term  “CRM”  conveys  a  negative  connotation  to  many  F-16  pilots  because  of  its  origins 
in  multicrew  aircraft.  Forty  percent  (40%)  of  the  respondents  indicated  that  they  did  not  like  the 
term  “CRM.”  While  94%  of  the  respondents  felt  that  CRM  principles  were  applicable  to  them, 
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33%  of  the  responses  on  attitudes  toward  F-16  CRM  training  related  to  CRM  not  being  a 
common  or  desired  point  of  reference  to  single-seat  fighter  pilots.  Sixteen  percent  (16%)  of  the 
suggestions  to  improve  CRM  training  recommended  using  another  term.  An  interesting  side 
note  was  that  pilots  who  had  previous  multicrew  fighter  experience  did  not  react  as  negatively 
toward  the  term  “CRM”  as  did  pilots  who  only  had  single-seat  fighter  experience.  In  similar 
perspectives,  91%  of  the  respondents  implied  that  they  were  already  doing  CRM  and  82%  said 
that  CRM  was  being  done,  but  it  was  not  called  “CRM.” 

2.  The  structure  of  the  CRM  training  was  important  to  many  F-16  pilots.  Forty-one  percent 
(41%)  of  the  suggestions  to  improve  the  training  focused  around  assuring  that  the  concept  of 
training  (delivery  format,  duration,  frequency,  etc.)  enhances  the  learning  process  and  is 
applicable  to  an  audience  of  single-seat  fighter  pilots.  Additionally,  CRM  training  does  not 
appear  to  be  integrated  into  F-16  simulator  training.  Twenty-three  of  the  interview  responses 
(1 1.3%)  indicated  that  there  was  no  CRM  training  conducted  in  the  F-16  simulators. 

3.  Personal  learning  styles  were  easily  self-identified  by  the  F-16  pilots.  Forty-nine  percent 
(49%)  assessed  themselves  to  be  “hands-on  learners,”  26%  thought  that  they  were  “visual 
learners,”  17%  felt  that  they  were  a  combination  of  “visual  and  auditory  learners,”  and  no  F-16 
pilot  (0%)  thought  that  they  were  “auditory  learners.”  While  most  F-16  pilots  (66%)  were  either 
a  hands-on  learner  or  a  combination  hands-on/visual  learner,  they  implied  that  most  of  their 
CRM  training  in  AETC  was  by  lecture,  using  auditor  and  visual  learning  formats. 

4.  CRM  training  is  not  consistent  within  the  F-16  community.  Eighty  percent  (80%)  of  F-16 
pilots  interviewed  had  not  had  CRM  specific  academics  in  Undergraduate  Pilot  Training.  This 
reflects  the  fact  that  CRM  training  has  only  been  introduced  over  the  past  few  years.  For  pilots 
who  did  not  receive  CRM  training  in  UPT,  CRM  initial,  instructor,  and  continuation  training  are 
also  provided  in  AETC  and  in  the  gaining  major  commands.  While  AETC  provides  CRM 
training  in  F-16  Formal  Training,  it  is  not  necessarily  in  the  same  format  as  that  which  is 
provided  during  CRM  continuity  training  in  the  gaining  major  commands. 

Research  Implications 
(From  overall  interviews) 

As  with  the  coding  of  the  data  and  topic  selection  for  data  reduction,  the  research 
implications  were  determined  by  the  principal  researcher,  a  former  F-16  pilot  and  fighter  wing 
commander. 

1.  Some  F-16  pilots  did  not  understand  the  breadth  of  the  issues  that  fall  under  the  overall 
umbrella  of  the  Air  Force  CRM  program  and  how  those  components  interrelate.  However,  they 
do  have  a  good  working  knowledge  of  the  individual  components  (USAF  CRM  behaviors 
highlighted  in  bold  print)  in  relationship  to  their  mission: 

Most  F-16  pilots  had  an  excellent  understanding  of  the  need  for  in-depth  mission 
planning/debrief.  Most  pilots  used  squadron  standards  for  delegating  mission  planning 
responsibilities. 
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Most  F-16  pilots  had  a  comprehensive  understanding  of  flight  (crew)  coordination/flight 
integrity  and  the  roles  of  flight  leader  and  element  leader,  but  there  were  variances  as  to  the 
wingman’s  role  in  communication.  Most  F-16  pilots  lean  toward  the  school  of  thought  where  the 
wingman  is  encouraged  to  speak  up  during  the  mission  planning  and  the  flight;  however,  there 
were  some  exceptions. 

Most  F-16  pilots  have  a  broad  understanding  of  risk  management/decision  making,  but 
there  was  some  variance  as  to  when  a  flight  leader  should  seek  assistance  from  outside  the  flight. 

Most  F-16  pilots  had  a  very  good  understanding  of  task  management  and  in-flight 
prioritization. 

Most  F-16  pilots  had  an  excellent  understanding  of  in-flight  communication  within  the 
four-ship  and  with  other  flights  and  outside  agencies. 

Most  F-16  pilots  had  a  very  good  understanding  of  situational  awareness  and  how  to 
maintain  it  or  regain  it. 

2.  F-16  instructor  training  (in  AETC)  does  not  appear  to  have  a  broad  CRM  component  that 
focuses  on  teaching  instructors  how  to  recognize  or  correct  shortfalls  in  the  students’ 
understanding  of,  and  implementation  of,  CRM  behaviors  and  skills.  If  the  instructors  are  to 
carry  the  responsibility  of  assuring  that  CRM  skills  and  behaviors  are  a  part  of  every  flight  and 
simulator,  then  those  instructors  should  have  frequent  CRM  refresher  training  to  highlight  recent 
innovations  and  developments  in  CRM  and  aviation  human  factors. 

3.  Pilots  appeared  to  prefer  interactive  CRM  training  with  problem-centered  facilitation,  in 
contrast  to  classroom  lecture.  Some  respondents  stated  a  preference  for  aircraft  and  mission  case 
studies  and  problem-solving  exercises.  Some  F-16  pilots  expressed  a  willingness  to  do  more 
training  in  simulator  missions  that  integrated  CRM  concepts,  but  not  as  “separate  CRM  events.” 
However,  pilots  who  had  used  the  ACC/US AFE  PC-based  situational  CRM  trainer  generally  did 
not  think  that  it  was  time-effective  because  it  took  too  long  to  familiarize  pilots  with  the  generic 
controls. 


Recommendations 

1.  Consideration  should  be  given  to  either  changing  the  term  “Cockpit/Crew  Resource 
Management”  (CRM)  to  something  less  offensive  to  single-seat  fighter  pilots,  or  minimizing  the 
use  of  that  term  with  single-seat  fighter  pilots  and  making  the  same  specific  CRM  training  more 
“transparent.” 

2.  AETC  and  the  gaining  operational  major  commands  should  stress  that  instructors  and 
evaluators  must  carry  the  CRM  banner  in  everyday  operations.  CRM  role  modeling  and  constant 
reinforcement  by  unit  instructors  and  evaluators  are  pivotal  to  the  retention  of  CRM  behaviors 
and  skills.  Instructors  and  evaluators  should  stress  CRM  behaviors  and  skills  on  every  flight  and 
in  every  simulator.  This  can  be  accomplished  in  a  “transparent  manner”  without  using  the  term 
“CRM;”  however,  all  of  the  CRM  behaviors  and  skills  must  still  be  covered. 


27 


3.  Instructor  and  evaluator  CRM  refresher  training  should  be  frequent  enough  to  insure  a 

heightened  focus  on  CRM  skills  and  behaviors  and  knowledge  of  the  recent  developments  in 
CRM  facilitation.  -  ...... 

4.  AETC  and  the  gaining  operational  major  commands  should  work  together  to  assure  that 
AETC  is  providing  aircraft/mission-specific  CRM  training  that  supports  the  gaining  commands’ 
requirements  and  that  the  gaining  major  commands  are  conducting  CRM  training  that  is  building 
on  the  format  and  structure  delivered  in  UPT  and  aircraft/mission-specific  formal  training. 
Consistency  of  behavior,  skills,  and  terminology  is  the  foundation  of  long-term  reinforcement, 
retention,  and  application. 

5.  CRM  continuation  training  (CT)  tfamfifg  for  AETC  training  units,  as  well  as  operational 
units,  must  underscore  CRM  behaviors  and  skills,  as  outlined  in  AFI  11-290,  Cockpit/Crew 
Resource  Management  Training  Program,  to  assure  that  pilots  are  aware  of  the  depth  of  the 
CRM  issues  and  the  breadth  of  CRM  interrelationships,  not  only  within  their  own  flying 
operation,  but  also  organizations  external  to  the  flying  units,  such  as  air  traffic  control  and 
maintenance. 

6.  CRM  training  should  use  facilitation  of  discussion  to  capitalize  on  adult  learning  models  and 
to  minimize  “lecturing,”  while  using  interactive  computer-based  training  (CBT)  to  stimulate 
class  dialogue.  Nonattribution  discussion  and  modeling  by  respected  peers  within  the  class  are 
highly  effective  reinforcement  tools.  An  experienced  facilitator,  with  mission/aircraft  credibility 
and  experience,  should  be  required  to  perform  this  important  task. 

7.  Formal  and  continuation  academic  training  should  incorporate  interactive,  hands-on,  visual, 
and  auditory  delivery  methods  to  include  aircraft-  and  mission-specific  case  studies,  CBT,  and 
video  reenactments  of  good  and  bad  examples  of  CRM. 

8.  Formal  and  refresher/continuation  CRM  training  should  be  structured  to  incorporate 
immediate  hands-on  application  immediately  after  CRM  academic  training.  Distributed  Mission 
Training  would  be  the  most  reinforcing,  if  facilitated  in  the  debriefing  by  a  qualified  CRM 
instructor.  If  DMT  is  not  available,  MDS  simulators  should  be  used  on  training  missions,  with 
scenarios  designed  to  create  CRM  “events”  which  must  be  addressed  (either  by  the  pilot  acting 
as  lead,  wingman,  or  part  of  a  crew).  These  scenarios  would  require  the  simulator  instructor  to 
perform  multiple,  scripted  roles  of  individuals  outside  the  cockpit,  such  as  flight  leader, 
wingman,  air  traffic  control,  etc. 

9.  If  Distributed  Mission  Training  or  simulator  missions  with  CRM  scenarios  are  not  available, 
consideration  should  be  given  to  developing  an  MDS-specific,  PC-based  flight  training  device, 
with  embedded  self-generating  CRM  scenarios,  for  use  as  a  CRM  procedural  trainer  with  a 
facilitated  debriefing. 

10.  Adopt  an  Integrated  CRM  Learning  Model,  using  adult  learning  principles,  cooperative 
group  learning,  and  learning  style  theory,  including  immediate  hands-on  application,  to  enhance 
reinforced,  long-term  retention  and  application  of  CRM  principles,  behaviors,  and  skills  (see 
Figure  1). 
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11.  Conduct  further  research  to  determine  the  validity  of  single-seat  fighter  pilot  statements 
regarding  CRM.  While  this  study  focused  on  personal  attitudes,  the  next  step  would  be  to 
conduct  research  using  direct  observation  of  pilot  CRM  behavior  during  briefing,  flight 
operations,  and  debriefing.  This  observation  research  will  help  identify  three  significant  factors: 
(a)  how  well  single-seat  fighter  pilot  statements  about  their  attitudes  toward  CRM  represent  the 
actual  use  of  CRM  in  everyday  operations,  (b)  key  CRM  behaviors  of  highly  effective  single¬ 
seat  fighter  pilots,  and  (c)  which  specific  CRM  skills  and  behaviors  should  receive  the  most 
emphasis  during  instructor  and  continuation  training  because  of  the  uniqueness  of  each  aircraft 
community.  This  type  of  research  has  been  successfully  performed  with  multicrew  aircraft 
(Silverman,  Spiker,  Tourville,  &  Nullmeyer,  1997),  but  has  not  yet  been  attempted  with  fighters. 

12.  Develop  a  quantitative  survey  from  the  responses  of  the  F-16  pilots’  interviews.  Administer 
this  survey  to  selected  F-16  units  (active  duty  and  Air  Force  Reserve  and  Air  National  Guard). 
Using  a  validated  F-16  CRM  survey,  develop  a  generalized  fighter  survey  to  be  administered  to 
other  representative  fighter  units  in  the  Combat  Air  Force  to  assess  the  depth  and  breadth  of 
application  and  knowledge  of  CRM  behaviors  and  skills  and  then  develop  subsequent 
modifications  of  the  CRM  training  to  assure  the  desired  knowledge  and  application  level. 

Summary 

This  research  was  initiated  to  examine  F-16  pilots’  attitudes  toward  CRM  and  to  solicit  the 
pilots’  recommendations  to  improve  CRM  training.  The  researchers’  intend  to  broaden  the 
application  of  this  study  to  all  single-seat  fighters  through  further  quantitative  research. 
However,  in  addition  to  the  statistical  presentations  and  interpretations  in  this  report,  the  F-16 
pilots’  expanded  comments  in  the  appendices  already  offer  a  rich  source  of  narrative  and 
quotations  that  should  be  helpful  in  understanding  the  underlying  foundational  attitudes  of 
single-seat  fighter  pilots  toward  CRM,  as  well  as  provide  insight  into  potential  enhancements  for 
CRM  training  for  the  entire  fighter  pilot  community. 
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APPENDIX  A 


Respondent  Experience  (By  interview  reference  number) 


Breakout  by  Rank  (n  =  36): 


Lt.: 

n  =  6 

16.7% 

Capt. 

1! 

36.1% 

Maj.: 

n  =  13 

36.1% 

Lt.  Col 

n  =  4 

11.1% 

Breakout  by  Interview  Reference  Number 


Number: 
Flying  Exp: 

1 

Total  Time: 
F-16: 

Other: 

Rank:  Captain  Position:  Flight  Commander 

2700  hours 

1300  hours  total/400  instructor 

F-4/T-38 

Number: 
Flying  Exp: 

2 

Total  Time: 
F-16: 

Other: 

Rank:  Captain 
1750  hours 

1 500  hours 

Position: 

Academic  Instructor 

Number: 
Flying  Exp: 

3 

Total  Time: 
F-16: 

Other: 

Rank:  Captain 
2000  hours 

550  hours 

T-37  IP 

Position: 

Squadron  IP 

Number: 
Flying  Exp: 

4 

Total  Time: 
F-16: 

Other: 

Rank:  Captain 
2000  hours 

800  hours 

T-38  IP 

Position: 

Squadron  IP/Stan  Eval 

Number: 
Flying  Exp: 

5 

Total  Time: 
F-16: 

Other: 

Rank:  IstLt 

300  hours 

60  hours 

Position: 

Student 

Number: 
Flying  Exp: 

6 

Total  Time: 
F-16: 

Other: 

Rank:  Captain 
1200  hours 

800  hours 

Position:  Squadron  IP/Scheduler 

Number: 
Flying  Exp: 

7 

Total  Time: 
F-16: 

Other: 

Rank:  IstLt 

300  hours 

60  hours 

Position: 

Student 
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Number: 

8 

Rank:  Major  Position:  Squadron  Assist  Ops  Officer 

Flying  Exp: 

Total  Time: 

2500  hours 

F-16: 

1 800  hours 

Other: 

AT-38  IP:  500  hours 

Number: 

9 

Rank:  2nd  Lt  Position:  Student 

Flying  Exp: 

Total  Time: 

300  hours 

F-16: 

Other: 

60  hours 

Number: 

10 

Rank:  IstLt  Position:  Student 

Flying  Exp: 

Total  Time: 

300  hours 

F-16: 

Other: 

60  hours 

Number: 

11 

Rank:  Captain  Position:  Student 

Flying  Exp: 

Total  Time: 

2560  hours 

F-16: 

65  hours 

Other: 

C-130:  2400  hours 

Number: 

12 

Rank:  Major  Position:  Squadron  IP 

Flying  Exp: 

Total  Time: 

2500  hours 

F-16: 

420  hours 

Other: 

F-4D/E/G:  1500  hours;  T-38  IP 

Number: 

13 

Rank:  Major  Position:  Squadron  Asst  Ops  Officer 

Flying  Exp: 

Total  Time: 

3000  hours 

F-16: 

1 500  hours 

Other: 

A- 10:  600  hours;  AT-38  IP:  700  hours 

Number: 

14 

Rank:  Major  Position:  Flight  Commander 

Flying  Exp: 

Total  Time: 

2400  hours 

F-16: 

800  hours 

Other: 

A-10:  1000  hours;  AT-38:  300  hours 

Number: 

15 

Rank:  Major  Position:  Squadron  IP/Safety 

Flying  Exp: 

Total  Time: 

3300  hours 

F-16: 

1300  hours 

Other: 

T-33:  1400  hours;  AT-38:  300  hours 

Number: 

16 

Rank:  IstLt  Position:  ANG  Student 

Flying  Exp: 

Total  Time: 

300  hours 

F-16: 

Other: 

60  hours 
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Number: 

17 

Rank:  Captain  Position:  Student 

Flying  Exp: 

Total  Time: 

1100  hours 

F-16: 

75  hours 

Other: 

B-l:  650  hours 

Number: 

18 

Rank:  Major  Position:  Student 

Flying  Exp: 

Total  Time: 

2000  hours 

F-16: 

5  hours 

Other: 

F-15:  1200  hours;  B-52Nav:  1200  hours 

Number: 

19 

Rank:  Major  Position:  TX  course  to  be  Assist  Ops 

Officer  at  Spangdahlem 

Flying  Exp: 

Total  Time: 

2800  hours 

F-16: 

1 000  hours 

Other: 

A- 10:  1000  hours;  AT-38  IP:  500  hours 

Number: 

20 

Rank:  Lt  Col  Position:  TX  course  to  be 

squadron  leader 

Flying  Exp: 

Total  Time: 

3500  hours 

F-16: 

1300  hours 

Other: 

F-15:  1000  hours;  MIG  29:  300  hours;  F-5:  500  hours 

Number: 

21 

Rank:  Captain  Position:  Student  going  to 

Thunderbirds 

Flying  Exp: 

Total  Time: 

2300  hours 

F-16: 

10  hours 

Other: 

A-10:  1400  hours;  F-15E  WSO:  650  hours 

Number: 

22 

Rank:  Major  Position:  Chief  of  Training  (Belgium 

Air  Force  Exchange) 

Flying  Exp: 

Total  Time: 

2500  hours 

F-16: 

Other: 

2000  hours 

Number: 

23 

Rank:  LtCol  Position:  Chief  of  Wing  Programming 

Flying  Exp: 

Total  Time: 

2500  hours 

F-16: 

970  hours 

Other: 

F-15:  1200  hours;  F-4  WSO:  500  hours 

Number: 

24 

Rank:  Major  Position:  Sqdn  IP/Chief 

Stan  Eval 

Flying  Exp: 

Total  Time: 

2300  hours 

F-16: 

Other: 

1950  hours 
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Number: 

25 

Rank:  Captain  Position:  Academic  Instructor 

Flying  Exp: 

Total  Time: 

1400  hours 

F-16: 

Other: 

1 100  hours 

Number: 

26 

Rank:  Captain  Position:  Flight  Commander 

Flying  Exp: 

Total  Time: 

1 800  hours 

F-16: 

Other: 

1500  hours 

Number: 

27 

Rank:  Major  Position:  Flight  Commander 

Flying  Exp: 

Total  Time: 

1300  hours 

F-16: 

1000  hours 

Other: 

F-4D/E/G  EWO:  500  hours 

Number: 

28 

Rank:  Lt  Col  Position:  Ops  Off  for  Range  Group 

Flying  Exp: 

Total  Time: 

F-16: 

350  hours 

Other: 

F-l  1  l/A-7/Tornado 

Number: 

29 

Rank:  Major  Position:  Asst  Ops  Off  for  Range  Gp 

Flying  Exp: 

Total  Time: 

F-16: 

1600  hours 

Other: 

OV-10:  300  hours,  T-37  IP 

Number: 

30 

Rank:  2nd  Lt  Position:  Student 

Flying  Exp: 

Total  Time: 

300  hours 

F-16: 

Other: 

31  hours 

Number: 

31 

Rank:  Captain  Position:  Squadron  IP/Scheduler 

Flying  Exp: 

Total  Time: 

1000  hours 

F-16: 

Other: 

700  hours 

Number: 

32 

Rank:  Major  Position:  Squadron  Chief  of  Stan  Eval 

Flying  Exp: 

Total  Time: 
F-16: 

Other: 

A- 10:  1000  hours;  UH-60:  900  hours  (special  ops) 

Number: 

33 

Rank:  Lt  Col  Position:  ORI  Prep;  Projected  Ops  Off 

Flying  Exp: 

Total  Time: 

2500  hours 

F-16: 

600  hours 

Other: 

F-4G:  1600  hours 
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Number: 

34 

Rank:  Major  Position:  Squadron  Assist  Ops  Officer 

Flying  Exp: 

Total  Time: 

2500  hours 

F-16: 

700  hours 

Other: 

F-4:  730  hours;  F-l  17:  275  hours;  AT-38:  275  hours 

Number: 

35 

Rank:  Captain  Position:  Student 

Flying  Exp: 

Total  Time: 

2000  hours 

F-16: 

900  hours 

Other: 

T-38  IP;  Test  Pilot  School 

Number: 

36 

Rank:  Captain  Position:  IP/Sqd  Stan  Eval 

Flying  Exp: 

Total  Time: 

2400  hours 

F-16: 

1350  hours 

Other: 

OV-10:  800  hours 

APPENDIX  B 


Respondent  Experience 
(By  pilot  category) 


Breakout  by  pilot  category 

Instructors:  14  39% 
Students:  11  30.5% 

Leaders:  11  30.5% 


Instructors:  14 

Number: 
Flying  Exp: 

1 

Total  Time: 
F-16: 

Other: 

Rank:  Captain  Position:  Flight  Commander 

2700  hours 

1300  hours  total/400  instructor 

F-4/T-38 

Number: 
Flying  Exp: 

2 

Total  Time: 
F-16: 

Other: 

Rank:  Captain 
1750  hours 

1500  hours 

Position:  Academic  Instructor 

Number: 
Flying  Exp: 

3 

Total  Time: 
F-16: 

Other: 

Rank:  Captain 
2000  hours 

550  hours 

T-37  IP 

Position:  Squadron  IP 

Number: 
Flying  Exp: 

4 

Total  Time: 
F-16: 

Other: 

Rank:  Captain 
2000  hours 

800  hours 

T-38  IP 

Position:  Squadron  IP/Stan  Eval 

Number: 
Flying  Exp: 

6 

Total  Time: 
F-16: 

Rank:  Captain 
1200  hours 

800  hours 

Position:  Squadron  IP/Scheduler 

Number: 
Flying  Exp: 

12 

Total  Time: 
F-16: 

Other: 

Rank:  Major  Position:  Squadron  IP 

2500  hours 

420  hours 

F-4D/E/G:  1500  hours;  T-38  IP 

Number: 
Flying  Exp: 

14 

Total  Time: 
F-16: 

Other: 

Rank:  Major 

2400  hours 

800  hours 

A-10:  1000  hours; 

Position:  Flight  Commander 

AT-38:  300  hours 

36 


Number:  15 

Rank:  Major 

Position:  Squadron  IP/Safety 

Flying  Exp:  Total  Time: 

3300  hours 

F-16: 

1300  hours 

Other: 

T-33:  1400  hours; 

AT-38:  300  hours 

Number:  24 

Rank:  Major 

Position:  Squadron  IP/Chief  Stan  Eval 

Flying  Exp:  Total  Time: 

2300  hours 

F-16: 

1950  hours 

Other: 

• 

Number:  25 

Rank:  Captain 

Position:  Academic  Instructor 

Flying  Exp:  Total  Time: 

1 400  hours 

F-16: 

1 100  hours 

Other: 

Number:  26 

Rank:  Captain 

Position:  Flight  Commander 

Flying  Exp:  Total  Time: 

1 800  hours 

F-16: 

1500  hours 

Other: 

Number:  27 

Rank:  Major 

Position:  Flight  Commander 

Flying  Exp:  Total  Time: 

1300  hours 

F-16: 

1000  hours 

Other: 

F-4D/E/G  EWO: 

500  hours 

Number:  31 

Rank:  Captain 

Position:  Squadron  IP/Scheduler 

Flying  Exp:  Total  Time: 

1 000  hours 

F-16: 

700  hours 

Other: 

Number:  36 

Rank:  Captain 

Position:  IP/Sqd  Stan  Eval 

Flying  Exp:  Total  Time: 

2400  hours 

F-16: 

1350  hours 

Other: 

OV-10:  800  hours 

Students:  11 

Number:  5 

Flying  Exp:  Total  Time: 
F-16: 

Other: 

Number:  7 

Flying  Exp:  Total  Time: 
F-16: 

Other: 


Rank:  IstLt  Position:  Student 
300  hours 
60  hours 


Rank:  IstLt  Position:  Student 

300  hours 
60  hours 
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Number: 
Flying  Exp: 

9 

Total  Time: 
F-16: 

Other: 

Rank:  2nd  Lt 

300  hours 

60  hours 

Position: 

Student 

Number: 
Flying  Exp: 

10 

Total  Time: 
F-16: 

Other: 

Rank:  IstLt 

300  hours 

60  hours 

Position: 

Student 

Number: 
Flying  Exp: 

16 

Total  Time: 
F-16: 

Other: 

Rank:  IstLt 

300  hours 

60  hours 

Position: 

ANG  Student 

Number: 
Flying  Exp: 

17 

Total  Time: 
F-16: 

Other: 

Rank:  Captain 
1100  hours 

75  hours 

B-l:  650  hours 

Position: 

Student 

Number: 
Flying  Exp: 

18 

Total  Time: 
F-16: 

Other: 

Rank:  Major  Position:  Student 

2000  hours 

5  hours 

F-15:  1200  hours;  B-52Nav:  1200  hours 

Number: 

Flying  Exp: 

21 

Total  Time: 
F-16: 

Other: 

Rank:  Captain 

2300  hours 

10  hours 

A- 10:  1400  hours; 

Position:  Student  going  to 
Thunderbirds 

F-15E  WSO:  650  hours 

Number: 
Flying  Exp: 

30 

Total  Time: 
F-16: 

Other: 

Rank:  2ndLt 

300  hours 

3 1  hours 

Position: 

Student 

Number: 
Flying  Exp: 

31 

Total  Time: 
F-16: 

Other: 

Rank:  Captain 
2560  hours 

65  hours 

C-130:  2400  hours 

Position: 

Student 

Number: 
Flying  Exp: 

35 

Total  Time: 
F-16: 

Other: 

Rank:  Captain  Position: 

2000  hours 

900  hours 

T-38  IP;  Test  Pilot  School 

Student 
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Leaders  (ADO  and  above):  11 


Number: 
Flying  Exp: 


Number: 
Flying  Exp: 


Number: 
Flying  Exp: 


Number: 
Flying  Exp: 


Number: 
Flying  Exp: 


Number: 
Flying  Exp: 


Number: 
Flying  Exp: 


Number: 
Flying  Exp: 


Number: 
Flying  Exp: 


8 

Total  Time: 
F-16: 

Other: 


Rank:  Major  Position:  Squadron  Assist  Ops  Officer 

2500  hours 

1800  hours 

AT-38  IP:  500  hours 


13 

Total  Time: 
F-16: 

Other: 


Rank:  Major  Position:  Squadron  Assist  Ops  Officer 
3000  hours 
1 500  hours 

A- 10:  600  hours;  AT-38  IP:  700  hours 


19 

Total  Time: 
F-16: 

Other: 


Rank:  Major  Position:  TX  course  to  be  Assist  Ops 
2800  hours  Officer  at  Spangdahlem 

1000  hours 

A-10:  1000  hours;  AT-38  IP:  500  hours 


20 

Total  Time: 
F-16: 

Other: 


Rank:  LtCol  Position:  TX  course  to  be  sqdn  leader 

3500  hours 
1300  hours 

F-15:  1000  hours;  MIG  29:  300  hours;  F-5:  500  hours 


22 

Rank:  Major 

Total  Time: 

2500  hours 

F-16: 

Other: 

2000  hours 

23 

Rank:  LtCol 

Total  Time: 

2500  hours 

F-16: 

970  hours 

Other: 

F-15:  1200  hours: 

Position:  Chief  of  Training  (Belgium 
Air  Force  Exchange) 

Position:  Chief  of  Wing  Programming 
F-4  WSO:  500  hours 


28  Rank:  LtCol  Position:  Ops  Off  for  Range  Group 

Total  Time: 

F-16:  350  hours 

Other:  F-lll/A-7/Tomado 


29  Rank:  Major  Position:  Asst  Ops  Off  for  Range  Gp 

Total  Time: 

F-16:  1600  hours 

Other:  OV-10:  300  hours;  T-37IP 


32  Rank:  Major  Position:  Squadron  Chief  of  Stan  Eval 

Total  Time: 

F-16: 

Other:  A-10:  1000  hours;  UH-60:  900  hours  (special  ops) 
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Number: 

33 

Rank:  Lt  Col  Position:  ORI  Prep;  Projected  Ops  Off 

Flying  Exp: 

Total  Time: 

2500  hours 

F-16: 

600  hours 

Other: 

F-4G:  1600  hours 

Number: 

34 

Rank:  Major  Position:  Squadron  Assist  Ops  Officer 

Flying  Exp: 

Total  Time: 

2500  hours 

F-16: 

700  hours 

Other: 

F-4:  730  hours;  F-l  17:  275  hours;  AT-38:  275  hours 
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APPENDIX  C 


Respondent  Distribution  by  Flying  Experience 


Line  IP  (n  =  14) 

F-16  only:  (2)  (6)  (24)  (25)  (26)  (31) 

F-16  +  1-seat  exp:  (14)  (15)  (36) 

F-16  +  2-seat  exp:  (12) 

F-16  +  multi-crew:  (1) 

F-16 +  Nav  exp:  (27) 

F-16  +  UPT  IP:  (3)  (4) 

Student (n  = 11) 

F-16  only:  (5)  (7)  (9)  (10)  (16)  (30) 

F-16  +  1-seat  exp: 

F-16  +  2-seat  exp: 

F-16  +  multi-crew:  (1 1)  (17)  (35) 

F-16 +  Nav  exp:  (18)  (21) 

F-16  +  UPT  IP: 

Leaders  (n  =  11) 

F-16  only:  (22) 

F-16  +  1-seat  exp:  (13)  (19)  (20)  (23)  (29)  (32) 
F-16  +  2-seat:  (28)  (33)  (34) 

F-16  +  multi-crew: 

F-16  +  Nav  exp: 

F-16  +  UPT  IP:  (8) 

Non-pilot: 
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APPENDIX  D 


CONSENT  FORM  FOR  INTERVIEW 
F-16  COCKPIT/CREW  RESOURCE  MANAGEMENT  (CRM)  RESEARCH 
Research  Study: 

You  are  invited  to  be  in  a  research  study  concerning  how  single-cockpit  F-16  pilots  apply 
Cockpit/Crew  Research  Management  (CRM)  in  daily  flight  operations.  We  ask  you  to  read  this 
form  and  ask  any  question  which  you  may  have  before  agreeing  to  be  in  the  study. 

This  study  is  sponsored  by  the  Warfighter  Training  Research  Division  of  the  Air  Force 
Research  Laboratory.  The  research  is  being  conducted  by  Dr.  Ron  Karp,  aviation  faculty  at 
Arizona  State  University  and  former  F-16  Vice  Commander  and  F-4G  Wing  Commander  (office: 
602-727-1873),  and  Dr.  Robert  Nullmeyer,  research  psychologist  at  the  Air  Force  Research 
Laboratory  (office:  602-988-6561). 

The  purpose  of  this  study  is  to  determine  F-16  pilots’  attitudes  toward  CRM,  as  well  as  the 
associated  skills  needed  for  application  of  CRM  knowledge.  The  results  of  this  study  will  be 
focused  on  impacting  future  CRM  training  so  that  training  can  be  tailored  to  meet  the  aircrews 
specific  needs. 

The  interview  concerning  your  experiences  and  understanding  of  CRM  should  be  less  than 
an  hour.  There  are  no  known  or  foreseen  risks  for  participation  in  this  study.  There  will  be  no 
monetary  compensation  for  this  study. 

Confidentiality: 

Your  name  will  not  be  tied  to  the  responses  of  the  interviews;  your  name  will  be  kept  on  a 
separate  list  only  to  prevent  duplication  of  interview  scheduling.  The  records  of  this  study  will 
be  kept  private  and  in  a  locked  file,  with  only  the  researchers  having  access  to  the  records.  In 
any  report  we  might  publish,  we  will  not  include  any  information  that  will  make  it  possible  to 
identify  the  individuals  interviewed.  You  will  be  given  a  copy  of  this  form  to  keep  for  your 
records. 

Statement  of  Consent: 

I  have  read  the  above  information  and  asked  any  appropriate  questions.  I  consent 
to  participate  in  the  study. 

Signature _ _ Date _ 

Signature  of 

Researcher _ Date  _ 
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APPENDIX  E 


INTERVIEW  WORKSHEET 

F-16  COCKPIT  RESOURCE  MANAGEMENT  (CRM)  RESEARCH 
Proposed  Research  Questions: 

1.  Experience/position/rank/instructor  or  student  status 

“Tell  me  about  your  background  and  experience.” 

Rank _ 

Current  Position_ _ _ _ 

Flying  Experience _ _ _ 

Background  (other  aircraft) _ _ _ 

2.  Learning  Style  (visual,  auditory,  or  hands-on) 

“How  do  you  think  you  learn  best?  Do  you  need  to  “see”  the  words  or  diagrams, 
or  do  you  need  to  “hear”  the  words,  or  do  you  need  to  “touch”  something  to  learn 
the  best?” 

3.  Overall  “attitude”  toward  CRM  training 

“What  type  of  formal  training  in  CRM  have  you  already  had?” 

“What  do  you  think  of  the  efforts  being  made  in  aviation  to  enhance  effectiveness 
and  improve  safety  by  emphasizing  CRM  training?” 

4.  Leadership  Emphasis 

“Do  you  feel  that  the  “top-down”  guidance  on  the  integration  of  CRM  into 
everyday  operations  is  clear?” 

Flight  and  Simulator  Integration 

“How  are  CRM  considerations  are  being  briefed,  employed,  and  debriefed  during  flight 
training  missions?” 

“What  about  during  simulator  training  missions?” 
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5.  Attitudes  on  Mission  Planning/Debrief 

“How  are  CRM  issues  covered  in  flight  planning  and  debriefing?” 

6.  Attitudes  on  Flight  Coordination/Flight  Integrity 

“What  role  do  you  think  a  flight  leader  should  take  in  a  four-ship  mission? 

“What  about  the  element  leader?” 

“What  role  should  a  wingman  take  in  a  four-ship  mission?” 

“At  what  point,  and  how,  does  a  wingman  question  or  non-concur  with  a  flight  leader’s 
action  or  decision?” 

7.  Attitudes  on  Risk  Management/Decision  Making 

“How  are  risks  assessed  and  decisions  made  in  a  four-ship  formation  different  than  those 
made  single-ship?” 

“At  what  point  does  a  flight  leader  or  ask  for  “outside”  recommendations/comments?” 

8.  Attitudes  on  Task  Management 

“How  does  a  flight  leader  distribute  tasks  and  workload  within  a  four-ship?” 

“How  does  an  individual  pilot  handle  task  and  workload  management?” 

9.  Attitudes  on  Communication 

“What  type  of  factors  have  you  observed  that  can  cause  a  breakdown  in  inter- flight 
communications?” 

“What  about  communication  breakdowns  with  other  flights  or  controlling  agencies?” 

10.  Attitudes  on  Situational  Awareness 

“Combat  and  training  missions  alike  are  very  complex.  From  your  perspective,  what  can 
a  flight  leader  do  to  improve  situational  awareness  in  a  four-ship?” 

“What  about  in  a  multi-force  package?”  (for  non-students) 

“What  about  as  an  individual  pilot?” 
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11.  Attitudes  on  Characteristics  of  Effective  Flight  Leaders  and  Wingman 

“What  characteristics  do  you  look  for  in  a  highly  effective  flight  leader?” 

“What  about  a  highly  effective  wingman?” 

12.  Attitudes  on  Instructors  and  Evaluators 

For  Students:  “Do  instructors  and  evaluators  use  CRM  principles  with  conviction  and 
enthusiasm  on  every  flight  and  simulator?” 

For  Instructors:  “Do  your  students  understand  the  importance  of  CRM  principles  being 
used  in  every  segment  of  flight  operations?” 

13.  Attitudes  on  Tactical  Relevance  of  CRM  Training 

“How  do  you  think  CRM  training  can  best  improve  your  personal  combat  effectiveness?” 
“Do  you  have  any  suggestions  on  how  to  improve  the  CRM  training?” 

15.  Researchers  Remarks: 
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APPPENDIX  F 


PRIMARY  DATA  COLLECTION 
(Interviews  with  F-16  Instructor  Pilots,  Leaders,  and  Students) 

Open  Coding  of  Data 

As  the  qualitative  data  were  collected,  recurring  themes  and  descriptive  words  were  used  to 
code  similar  topics  together  to  form  data  coding.  Coded  data  were  later  represented  in  various 
matrix  forms  to  aid  in  cross-referencing  the  data.  Using  grounded  theory,  the  data  analysis 
included  open  coding,  which  involved  determining  what  themes  and  categories  emerged  from 
direct  responses  from  the  F-16  IPs,  unit  leaders,  and  F-16  students  (Creswell,  1994).  The  key 
phrases  were  collected  and  coded  into  meaningful  categories.  All  key  phrases  were  considered 
for  use  in  the  narrative. 


Axial  Coding  of  Data 

After  completion  of  the  qualitative  data  collection  using  opened  coding  of  similar  topics, 
axial  coding  of  the  data  was  accomplished  by  looking  at  the  comparisons  and  interrelationships 
of  the  F-16  pilots  in  regard  to  CRM.  During  the  course  of  the  study,  the  topical  coding  of  the 
responses  evolved  and  was  recoded  as  necessary  to  depict  an  accurate  portrayal  of  F-16  pilot 
CRM.  Axial  coding  tabulations  are  reflected  in  the  findings  component  of  this  report 

Narrative  Format 

The  following  narrative  description  of  the  respondents’  remarks  is  arranged  in  axial  coding 
within  each  original  open  coding  topic.  The  parenthesis  following  each  response  refers  to  the 
demographical  identification  of  the  respondent  in  Appendix  A. 

CRM  Formal  Training 
Undergraduate  Pilot  Training  (UPT) 

•  At  UPT  at  Sheppard  AFB  in  1993,  received  CRM  component  in  prioritization  and  task 
management,  but  they  did  not  know  how  to  deal  with  single-seat  CRM  concept  (8  to  12 
hours).  (27) 

•  Navy  classes  involving  mostly  slide  shows.  One  hour  of  instruction  before  flying  T-34s,  and 
then  one  hour  at  the  end  of  flying.  The  Navy  also  used  some  “role-playing”  which  was  good. 
In  T-38  training,  1-hour  class  focused  on  multicrew  CRM  with  some  of  the  same  slides  the 
Navy  had  shown.  (5) 

•  Only  1  hour  in  T-3 7  academics.  Also  a  short  session  in  AT-38s.  Both  of  these  were  mostly 
pictures  of  aircraft  crashing.  (7) 

•  Total  of  three  hours  of  classes:  At  beginning  of  UPT  as  overview,  halfway  through  T-37s, 
and  at  end  of  T-38s.  Taught  by  physiology  training  people.  Mostly  general  information,  but 
were  shown  three  films  showing  crashes  with  good  and  bad  examples  of  CRM  (9) 

•  Five  hours  in  UPT  on  general  subjects  by  aviation  physiologists.  (10) 
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•  Once  in  a  pilot  meeting  in  UPT.  (18) 

•  In  T-37s/T-38s,  about  2  to  3  hours  of  lecture  on  risk  assessment,  CRM,  and  communication, 
using  mostly  videos  of  good  and  bad  CRM  and  mishaps.  (30) 

•  One  to  two  hours  in  Introduction  to  Fighter  Fundamentals  (IFF).  (30) 

•  Had  crew  coordination,  but  it  was  not  called  CRM.  (36) 

F-16  Formal  Training  (FT) 

•  “Classroom”  at  Luke,  4-hr  course,  in  generalized  physiological  training  and  situational 
awareness.  (1) 

•  In  FT,  CRM  training  was  tailored  to  practical  applications,  but  instructor  is  not  a  “true 
believer.”  One-hour  mass  briefing  to  “fill  the  square.”  (2) 

•  It  was  in  the  middle  of  all  other  academics.  (3) 

•  No  real  CRM  in  FT.  Simulator  IPs  don’t  give  any  help  during  EP  simulators.  (12) 

•  Received  a  briefing  by  safety  people  that  was  supposed  to  be  on  how  CRM  applied  to 
fighters,  but  it  really  showed  how  “CRM  is  for  multicrew  airplanes.”  (14) 

•  Short  lecture  at  Moody  in  squadron.  (26) 

•  Twice  in  training  at  Luke  (1993),  mostly  about  tasking  in  cockpit.  However,  some  people 
are  beginning  to  recognize  that  this  is  more  than  just  where  you  put  the  checklists  in  the 
cockpit.  (27) 

•  Course  in  “Human  Resource  Management”  which  had  some  CRM  and  lots  of  aviation 
physiology  like  “G-LOC.”  (5) 

•  One  hour  in  formal  training  at  Luke.  (7) 

•  Two  hours  at  Luke  on  general  subjects.  (10) 

•  Some  risk  assessment  at  Luke,  but  no  videos.  Instructor  is  an  aviation  physiologist  and  it 
would  be  better  if  the  instructor  was  a  pilot.  (30) 

•  In  TX  transition  course  (1991)  had  one  hour  in  air-to-air  and  one  hour  in  air-to-ground  from 
the  training  squadron  on  accidents  and  error  chain.  (35) 

•  Responsibilities,  emergencies,  etc,  in  1992.  (36) 

•  Principles  in  bits  and  pieces  during  training,  but  “not  as  CRM  specific.”  (8) 

•  Some  general  awareness  a  few  years  ago  called  “AAAMP”  which  covered  G-LOC  and 
situational  awareness.  (19) 

•  F-16  TX  pilots  are  given  a  shortened  version  of  what  the  B  course  pilots  get,  but  even  that  is 
not  specifically  called  CRM.  (19) 

•  Had  something  at  Hill  in  mid-90s,  but  didn’t  remember  what  it  was.  It  had  a  lecture  and  not 
the  ACC  CRM  van.  (29) 

•  At  Mountain  Home  (1992)  initial  training  was  by  physiology  instructor.  Only  10%  useful 
and  couldn’t  remember  what  was  taught.  (32) 

•  None:  (23)  (9)  (11) 

F-16  Instructor  Course  (I) 

•  “Luke  training  paled  in  comparison  to  the  dedicated  ACC  CRM  van  program.  ”(3) 

•  In  academics  at  Luke  we  talked  a  little  about  CRM,  but  it  was  mostly  a  “square-filler.”  (6) 

•  Discussions  included  how  CRM  relates  to  Luke.  (24) 
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•  Classroom  lecture  by  wing  aviation  physiologist  went  over  case  studies  for  about  2  hours. 
(25) 

•  We  only  had  a  “contingency  meeting”  with  squadron  commander  to  discuss  emergency 
procedures  and  how  to  resolve  problems.  (26) 

•  1  to  1  14  hours  at  Luke  on  common  errors  students  will  make.  (22) 

•  Lecture  by  physiologist  on  stress  and  what  to  look  for  in  students.  (33) 

•  None:  (23)  (29)  (27) 

F-16  Continuation  Training  (CT> 

Note:  Comments  are  ordered  by  MAJCOM  and  whether  the  comments  were  neutral,  strongly 
positive,  or  less  than  positive  experience. 

AETC. 


Neutral  (or  mixed)  experience  (n  =  12): 

•  “Classroom”  at  Luke,  1-2  hr  course  in  generalized  physiological  training  and  situational 
awareness.  (1) 

•  In  periodic  physiological  training,  with  G-LOC,  stress,  etc.  (1) 

•  Part  of  physiology  training,  but  was  mostly  risk  assessment  and  task  management.  About  1 
to  2  hours.  (4) 

•  CRM  training  in  operations  is  integrated  into  daily  flying  activities,  it  is  not  separated  as  a 
separate  subject.  (6) 

•  Once  a  year  at  Luke  in  a  classroom  for  a  couple  of  hours.  (25) 

•  One  hour  of  discussion  using  slides  and  videos  on  incidents  and  accidents  taught  wing 
aviation  physiologist.  (26) 

•  One  hour  at  Luke  talking  about  incidents  and  accident  prevention.  (31) 

•  In  ops  unit,  had  about  45  minutes  of  “misprioritization  that  leads  to  mishaps.”  (20) 

•  1  to  1  14  hours  per  year  in  squadron  at  Luke,  given  by  the  Academic  Squadron  aviation 
physiologist,  covering  incidents  and  accidents  and  the  thought  process  that  occurred.  This  is 
a  “grounding  item”  if  it  is  not  attended.  (22) 

•  Case  studies  once  a  year  by  wing  aviation  physiologist.  (28) 

•  Once  a  year  squadron  talks  about  risks.  (33) 

•  Only  general  terms  in  pilots’  meeting  and  safety  meetings.  (34) 

Less  than  positive  (n  =  1): 

•  One  hour  of  lecture  in  classroom:  “Everyone  hated  it.”  (32) 
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ACC/USAFE/PACAF  (n  =  8). 


Neutral  (or  mixed)  experience  (n  =  6): 

•  USAFE  CRM  van  in  1995  at  Aviano  (ACC  contractor)  was  useful.  It  is  a  good  idea,  but  it 
focused  heavily  on  when  you  should  speak  up.  Discussion  of  incidents  was  beneficial,  but 
the  use  of  the  small  flight  simulator  in  van  was  not  cost  effective.  (24) 

•  Two  days  in  classroom  at  Ramstein  (1992).  Mostly  lecture  and  discussion  about  accidents. 
(24) 

•  At  Hill  and  in  Korea.  At  Hill  in  ACC  CRM  van  in  late  1996.  What  they  offered  was  good, 
but  it  took  a  whole  day  and  we  were  already  doing  what  they  recommended.  (36) 

•  ACC  CRM  at  Moody  was  “decent  stuff,”  but  was  too  long  (4  to  5  hours).  Setting  up  the 
situational  trainer  (PC  flight  simulator)  took  too  much  time.  (3 1) 

•  At  Mountain  Home  in  1995  in  the  ACC  CRM  van.  First  couple  of  times  it  was  worthless 
because  it  had  a  heavy  focus  on  multiengine  CRM.  But  then  it  changed  to  move  to  more 
single-seat  focus  and  fighter  application  and  became  a  very  good  refresher.  PC  flight 
simulator  was  fun,  but  not  useful.  (33) 

•  ACC  mobile  CRM  van  at  Shaw  (twice).  Worthwhile  training,  but  it  was  a  little  long  because 
of  the  day-to-day  operational  tempo.  It  drives  things  like  shooting  the  gun  to  the  weekends. 
(4) 

Strongly  positive  experience  (n  =  6): 

•  ACC  “CRM  van”  at  Mountain  Home,  4-hr  course.  Hands-on  training  really  useful,  along 
with  case  studies  and  instructors  with  fighter  background.  (1) 

•  Tactical  scenarios  in  the  ACC  CRM  van  at  Mountain  Home  were  very  relevant.  (1 ) 

•  ACC  van  at  Mountain  Home:  First  time  they  were  just  getting  started  and  it  was  hit  and  miss, 
but  the  second  time  (1997)  was  better.  The  situational  trainer  was  good  to  let  young  guys 
know  how  they  fit  into  a  4-ship  picture.  (14) 

•  ACC  “CRM  van”  at  Hill  (twice).  It  had  “very  worthwhile  with  applicable  scenarios,  but  the 
situation  trainer  not  very  useful  for  continuation  training.”  (3) 

•  Six-hour  course  at  Spangdahlem  in  CRM  van  1996.  Case  studies  were  good,  but  CRM  or 
emergency  training  in  a  real  simulator  would  have  been  better  than  the  PC-based  simulators 
in  the  van.  (25) 

•  ACC  CRM  van  at  Davis-Monthan  (A- 10)  for  4  hours.  Case  studies  using  video  and  group 
discussion  were  excellent.  Sterile  environment  (no  interruptions)  creates  very  good 
atmosphere  to  learn  how  different  people  handle  CRM  issues.  Hands-on  tasks,  using  CBT 
and  computer  flight  simulator,  were  very  beneficial  to  emphasize  hands-on,  quick  decision 
making.  Really  liked  the  ACC  CRM  video  playback  capability  with  “comm”and  the 
experienced  instructors.  (21) 

Less  than  positive  experience  (n  =  6): 

•  Did  not  have  time  to  go  to  CRM  training  in  the  USAFE  CRM  van  at  Spangdahlem, 
Germany,  but  “heard”  from  friends  that  it  was  half  good,  half  bad.  The  bad  part  was  that  it 
was  too  “heavy  aircraft”  orientated.  (6) 
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•  Initial  CT  training  at  Shaw  in  ACC  CRM  van  for  4  lA  hours,  including  a  2-hour  hands-on 
trainer.  Some  of  the  exercises  were  very  useful  to  emphasize  CRM  issues.  However,  follow- 
on  CT  was  a  waste  of  time  for  single-seat  fighter  pilots  (except  for  F-15E  and  EF-llls) 
because  it  emphasized  too  much  “crew  coordination.”  (12) 

•  Took  ACC  initial  CRM  at  Shaw  in  the  van  but  it  “was  not  different  from  what  I  was  already 
doing:  brief-fly-debrief.  Didn’t  learn  anything  new.”  (13) 

•  “Saw”  ACC  CRM  van,  but  did  not  go  through  the  course  because  it  took  so  much  time.  (20) 

•  At  Kunson,  lecture  and  slides,  but  too  slide  show  oriented.  (28) 

•  Four  hours  at  Holloman  in  ACC  CRM  van  following  a  mission  profile  (1996).  “Not  very 
productive;  I  already  knew  that  I  was  supposed  to  use  all  of  my  resources.”  (18) 

•  None:  (23)  (29) 

Other  CRM  Formal  Training 

•  The  best  CRM  training  was  going  through  the  accident  investigation  school  at  the  Safety 
Center  at  Kirkland.  It  was  heavy  on  human  factors.  (6) 

•  As  a  former  navigator,  received  a  lot  of  crew  coordination  in  nav  school,  Holloman  WSO 
lead-in,  and  F-4  school.  (27) 

•  In  first  F-4  operational  unit  in  late  80s,  received  CRM  that  was  applicable  only  to  heavy, 
multicrew  aircraft.  (27) 

•  In  operational  F-16  unit,  had  a  general  lecture  once  a  year  along  with  all  the  other  briefings. 
(27) 

•  Twenty  hours  total  at  Kirkland,  Hurlbert,  and  Kadena.  Mostly  just  classroom  lecture  and 
slides.  (11) 

•  Old  “AAMP”  program  at  Seymour  Johnson  and  Shaw  was  only  “lip  service  to  CRM  done  by 
flight  surgeons.”  It  had  more  of  a  focus  on  flying  than  interpersonal  relationships.  (21) 

•  Some  CRM  at  centrifuge  training  on  task  management  and  psychological  training  (1.5 
hours).  (28) 

•  At  AF  Safety  School  at  Kirkland  (1997).  Reviewed  case  studies  of  accidents  and  listened  to 
recordings  and  read  findings.  (29) 

•  Talked  about  “crew  coordination”  when  in  F-4  ops  unit.  (34) 

•  Never  talked  about  CRM  in  F- 1 1 7.  (34) 

•  In  A- 10  Instructor  course,  had  a  1.5  hour  overview  of  CRM  concepts,  but  it  was  not 
beneficial  for  IP  duties.  (21) 

Attitudes  Toward  F-16  CRM  Training  Program 

USAF  emphasis  on  CRM 

F-16  pilots  feel  that  they  are  already  doing  CRM. 

•  CRM  is  already  in  place  for  single-seat  fighter,  in  regards  to  training  rules,  risk  assessment, 
briefing  to  lowest  common  denominator,  and  weather  considerations.  (1) 

•  “It  can’t  hurt,  but  it  is  not  that  beneficial  nor  will  it  help  save  an  accident  because  we  do  it 
already.”  (24) 
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•  “In  the  single-seat  world,  CRM  has  always  been  done,  like  prioritization  and  where  to  put 
things.  (25) 

•  “I  don’t  know  if  it’s  useful  training.”  (25) 

•  Fighter  pilots  have  always  trained  to  prioritize  tasks  and  use  coordination,  but  under  different 
names.  The  use  of  additional  new  CRM  terminology  causes  some  concern.  (4) 

•  Some  of  the  things  in  CRM  training  you  can  use,  but  most  is  just  common  sense.  (31) 

•  It  is  already  being  done.  (5) 

•  Everything  is  already  laid  out  already  for  single-seat  fighter  (through  the  “contract”)  so  there 
is  no  need  to  training  to  do  CRM.  (29) 

•  CRM  is  something  you  do  everyday,  but  cannot  define.  (32) 

•  We’ve  been  doing  CRM  all  along,  but  we  haven’t  called  it  CRM.  (36) 

•  There  is  a  lot  of  resistance  to  CRM  training  because  the  fighter  community  has  been  doing 
this  for  years;  it  is  just  that  they  did  not  call  it  anything.  (33) 

F-16  pilots  feel  that  they  are  being  “forced”  to  do  CRM  training. 

•  Just  “do”  CRM,  do  not  “invoke”  it.  (1) 

•  “CRM  training  is  something  that  you’re  forced  to  do  to  fill  a  square.”(2) 

•  If  leadership  says:  “Here’s  CRM,  just  go  do  it,  it  doesn’t  work!  The  more  in-tuned  the 
leadership  is  with  what’s  going  on  on  the  flight  line,  the  better  CRM  guidance  will  be  given.” 
(6) 

•  “I  don’t  like  things  that  are  ‘imposed’  upon  pilots.  CRM  is  too  forced.”  (12) 

•  “CRM  is  the  official  way  to  cover  the  Air  Force’s  butt.”  (24) 

•  Very  negative  opinion.  “It’s  just  another  one  of  those  things  we  have  to  do;  the  latest 
management  de  jour.”  (20) 

Use  of  the  term  “CRM”  is  not  a  common  or  desired  point  of  reference  for  F-16  pilots. 

•  CRM  is  something  that  is  aligned  with  a  crew  concept  and  is  not  applicable  to  single-seat 
fighter  pilots,  but  “CRM  concepts”  are  ok  and  applicable.  (2) 

•  “Never  refer  to  CRM  as  ‘CRM’  because  it  will  turn  people  off!”  (14) 

•  CRM  is  not  a  term  that  is  used  or  talked  about.  (7) 

•  In  heavies,  CRM  covered  safety,  crews  speaking  up,  getting  people  to  speak  up.  Not  much 
CRM  in  fighters.  (11) 

•  At  Luke,  CRM  principles  are  being  given  in  each  flight,  but  not  referring  to  them  as  CRM. 

(21) 

•  “Anything  we  can  do  to  lower  the  mishap  rate  or  help  the  new  kids  is  important,  but  it 
matters  what  you  call  it.”  Do  not  call  it  “CRM.”  (18) 

•  CRM  is  for  multiplace  aircraft,  not  F-16.  (23) 

•  The  name  CRM  probably  needs  to  be  changed  because  of  a  lot  of  pilot's  sensitivity  to  it.  (28) 

•  CRM  is  applicable  to  crew  aircraft,  like  F-15E  and  C-141,  but  it  is  not  applicable  to  single¬ 
seat  fighter.  (29) 
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F-16  CRM  training  should  focus  on  single-seat,  not  multicrew,  aircraft. 


“CRM  is  not  really  being  addressed  toward  a  single-seat  audience.”  (14) 

Effort  is  good,  but  CRM  needs  to  be  specialized  for  single-seat  aircraft  using  flight 
communication  techniques,  instead  of  using  multiplace  airplanes  and  flight  deck 
communication  procedures.  (35)  (36) 

Needs  to  be  single-seat  specific.  (1) 

From  a  single-seat  pilot’s  view,  CRM  is  not  really  applicable  to  the  fighter  world.  (33) 

USAF  needs  to  show  that  it  is  serious  about  CRM. 

“Great  reluctance  of  the  captain-on-the-street  to  adopt  something  unless  they  see  that  the  Air 
Force  is  serious  about  implementing  it.”  (14) 

Not  sure  people  know  what  CRM  is.  Some  like  it;  some  think  that  it  is  bunk.  There  is  a  lack 
of  dollars  to  support  the  CRM  program,  but  “we  talk  a  good  game.”  (1 5) 

The  program  is  evolving  and  getting  better.  It  is  worthwhile,  but  we  do  not  enjoy  doing  it.  If 
it  is  not  formalized  and  required,  then  people  will  not  do  it.  (27) 

Other  positive  comments. 

“ACC  program  is  a  vast  improvement  over  past  programs.”  (3) 

While  people  have  not  been  responsive  to  CRM  in  the  past,  they  seem  to  be  getting  better. 

(10) 

Much  more  concentrated  effort  now  because  it  is  being  taught  by  experienced  aircrews,  not 
flight  surgeons.  (21) 

Studied  CRM  for  airlines  in  college,  but  now  sees  the  benefits  to  fighters.  (30) 

CRM  is  valuable  because  it  encourages  students  to  speak  out.  (30) 

“Formal  program  is  a  step  in  the  right  direction.”  (19) 

“Anything  that  will  improve  safety  and  not  degrade  combat  capability  is  good.”  (36) 

CRM  training  is  useful  if  it  is  done  the  right  way  and  not  just  filling  a  square.  It  should  be 
more  frequent  and  not  just  annually.  (34). 

Since  CRM  deals  with  emergency  procedures,  it  should  be  discussed  during  the  “emergency 
of  the  day”  in  the  flight  briefing.  (26) 

Other  negative^  comments. 

“It’s  wasting  my  time.  I  am  not  a  big  advocate  of  CRM.”  (13) 

Every  two  or  three  years  the  Air  Force  comes  up  with  a  new  “program  to  worship.”  (36) 

After  the  first  question  about  the  efforts  being  made  in  the  Air  Force  to  enhance  effectiveness 
and  improve  safety  by  emphasizing  CRM  training,  the  respondent  said  to  interviewers  (in  a 
harsh  manner):  “You  tell  me,  just  what  does  the  Air  Force  intend  to  do  with  CRM....”  (32) 

If  CRM  training  takes  a  long  time,  single-seat  fighter  pilots  will  go  to  it  with  a  negative 
attitude.  “The  shorter  you  can  make  continuation  training,  one  hour  desired,  the  better.”  (31) 
“I  came  out  of  a  staff  tour  and  I’m  not  really  up  to  speed  on  CRM.”  (8) 


There  is  reluctance  for  people  to  speak  up  because  they  might  sound  stupid.  If  it  is  a  safety 
of  flight  issue,  then  they  should  speak  up.  (29) 


Integration  of  CRM  into  everyday  operations  ( To^down  guidance  to  pilots) 

F-16  pilots  feel  that  they  are  already  doing  CRM. 

•  We  know  about  CRM  “once  in  a  while  by  hearing  a  buzzword,  but  it’s  really  a  different 
name  for  something  we’ve  already  been  doing.”  (25) 

•  “It’s  just  part  of  our  job  and  it’s  something  that  doesn’t  require  any  direction.”  (32) 

•  Fighter  pilots  already  do  CRM,  and  it’s  “common  sense.”  (36) 

F-16  pilots  feel  that  they  are  being  “forced”  to  do  CRM. 

•  “Everyone  was  told  that  they  “would  go  to  CRM  training”  and  it  was  just  put  on  the 
schedule.”(3) 

•  Message  is  clear  that  we  “have  to  do  this,  but  the  real  reason  we  have  to  do  it  is  to  fly.”  (14) 

•  It’s  something  that  you  just  have  to  do;  just  “salute  and  do  it.”  (20) 

•  It  is  apparent  that  it  is  a  command-wide  directive  that  must  be  taught.  (33) 

•  Guidance  is  clear  that  “higher-ups”  want  it  done.  (19)  (34) 

Use  of  the  term  “CRM”  is  not  a  common  or  desired  point  of  reference  for  F-16  pilots. 

•  Squadron  commander  is  into  it,  but  he  does  not  reference  the  word  “CRM”  as  a  separate 
issue.  (1)  (6) 

•  Never  any  “direct  guidance  on  CRM,”  but  they  talk  about  CRM  issues.  (2) 

•  “Absolutely  coming  down,  but  not  specifically  under  the  ‘CRM’  label.”  (4)  (14) 

•  Only  in  regards  to  using  common  sense  and  all  your  resources.  But  people  do  not  say  the 
word  “CRM.”  (31) 

•  “Someone  at  the  top  thinks  it’s  important.  It’s  not  just  filling  a  square,  but  it’s  really  not  for 
us,  it’s  for  crew  planes.”  (5) 

•  “I  never  heard  the  word  CRM  used  once  at  Luke.”  Everything  comes  down  to  “lead”  and 
“wingmen.”  (7) 

•  “Don’t  ever  hear  the  word  “CRM”  used,  but  they  do  discuss  things  like  “comm  drill”  and 
“don’t  be  afraid  to  tell  me  if  you  see  something  screwed-up.”  (9) 

•  CRM  principles  are  encouraged,  but  there  is  no  formal  CRM  discussion.  (1 1)  (1 8) 

•  CRM  never  mentioned  specifically,  but  some  issues  are  covered  in  flights  and  classes.  (30) 

•  In  commander’s  call  a  lot  of  “safety  issues”  are  discussed,  but  not  especially  as  CRM  issues. 

(22) 

•  The  message  is  given  in  the  multicrew  aircraft,  but  not  in  the  single-seat  community.  (29) 

•  Squadron  Commander  and  Operations  Officer  stresses  some  CRM  subjects,  but  never 
mention  the  word  CRM.  (14)  (24) 
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F-16  CRM  training  should  focus  on  single-seat,  not  multicrew,  aircraft. 


•  No  applicable  responses. 

USAF  needs  to  show  that  it  is  serious  about  CRM. 

•  The  Air  Force  has  placed  emphasis  on  CRM,  but  it  is  not  clear  in  the  wing  or  squadron  as  to 
the  specific  direction.  (35) 

Other  positive  comments. 

•  Mentoring  is  very  important  for  the  IPs  to  show  students  the  importance  of  CRM 
concepts.(l) 

•  AETC  stresses  “rules”  and  the  IPs  emphasize  them.  “CRM  helps  give  you  ‘common  sense’ 
when  you  don’t  have  any.”  (12) 

•  There  has  been  a  high  emphasis  in  the  wing,  because  of  the  nature  of  the  business  and 
potential  emergencies,  to  incorporate  it  into  the  briefings.  (26) 

•  “Absolutely  -  it  has  to  be  from  the  top.”  (2 1 ) 

•  It  is  clear  that  “the  threat  is  out  there  all  the  time,  so  you  can’t  let  your  vigilance  down.”  (21) 

•  Use  all  facilities  and  resources  at  your  disposal.  (23) 

Other  negative  comments. 

•  There  are  so  many  projects  going  on  all  the  time,  that  people  only  have  time  to  focus  on  the 
current  hot  topics.  (27) 

•  “People  think  CRM  is  a  good  tool,  but  hard  to  implement.”  (10) 

•  Older  guys  “turn-off’  phrases  like  ORM  and  CRM,  but  components  are  discussed.”  (8) 

•  There  s  a  resistance  at  the  operator  level  to  implement  CRM  because  of  its  background  in 
multicrew  aircraft.  (36) 

•  Hard  to  do  because  “all  the  paper  work  and  flying  that  needs  to  be  done  in  a  long  duty  day; 
but  it  is  OK  if  it  is  a  good  program.”(3) 

•  In  ACC  it  was  clear  guidance.  In  PACAF  there  was  no  guidance.  In  AETC,  it’s  not  really 
talked  about  specifically.  (13) 

Integration  of  CRM  into  everyday  operations  (Integration  into  flight  operations!. 

F-16  pilots  feel  that  they  are  already  doing  CRM. 

•  “CRM  has  always  been  done  in  the  single-seat  world,  that’s  why  single-seat  guys  ‘cut-up’  on 
CRM.”  (21) 

•  Difficult  to  implement  as  a  specific  effort  because  some  CRM  is  interwoven  into  everything 
we  do,  like  mission  planning,  support  in  flight,  and  debrief.  (8) 
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F-16  pilots  feel  that  they  arc  being  “forced”  to  do  CRM. 

•  Principles  should  be  applied,  but  no  one  wants  another  line  in  the  briefing  guide  that  says: 
“CRM.”  (19) 

Use  of  the  term  “CRM”  is  not  a  common  or  desired  point  of  reference  for  F-16  pilots. 

•  CRM  is  integrated  into  mission  planning  and  flight  operations,  but  not  mentioned  as  a 
separate  topic.  (1)  (14) 

•  IPs  don’t  talk  specifically  about  CRM,  but  they  talk  about  student’s  cockpit,  where  the 
student  fits  into  the  flight,  rules  and  responsibilities  of  all  flight  members,  prioritization  of 
responsibilities,  and  task  saturation.  (2). 

•  CRM  is  not  talked  about  directly  as  CRM,  but  it  is  filtering  down  that  single-seat  fighter 
pilots  “have  to  talk  to  others  in  the  system”  (SOF,  Top-Three,  etc).  (3) 

•  CRM  is  never  discussed  as  a  specific  subject,  but  some  CRM  considerations  are  integrated. 

(4) 

•  “Heavy  guys  need  CRM  because  the  different  people  in  the  crew  are  all  thinking  about 
separate  issues  and  do  not  see  the  same  big  picture.  Fighter  guys  need  to  get  everyone  to  be 
able  to  think  independently  and  always  see  the  big  picture.”  (6) 

•  “CRM  is  done  better  by  fighter  pilots  than  anyone  else,  but  we  don’t  think  of  ourselves  as 
crews.”  (6) 

•  Principles  and  concepts  are  integrated  in  everyday  life,  but  not  specifically  as  CRM.  (25) 

•  No  one  ever  talks  specifically  about  CRM.  CRM  is  not  done  intentionally,  it  is  “just  a  by¬ 
product  of  what  we  do  naturally.”  27) 

•  It  is  being  implemented  and  covered,  but  it’s  not  called  CRM  because  “CRM  is  for  multi¬ 
crew  airplanes.”  (5) 

•  Although  CRM  principles  are  discussed,  “CRM  is  never  mentioned  by  name.”  (7)  (1 8) 

•  The  IPs  talk  about  it,  but  not  specifically.  (10) 

•  It  is  done  on  regular  basis,  but  it  will  not  be  labeled  as  CRM.  (21) 

•  Never  hear:  “we’re  going  to  do  some  CRM  today....”  (30) 

•  CRM  is  part  of  an  everyday  briefing,  but  not  as  a  CRM  topic.  (35) 

•  Core  principles  are  probably  followed,  but  hardly  ever  hear  the  word  “CRM.”  (23) 

•  Almost  everything  is  briefed  and  conducted  in  accordance  with  the  squadron  standards,  but 
they  do  not  focus  on  CRM  issues  and  they  do  not  ever  call  it  “CRM.”  (33) 

•  CRM  considerations  probably  take  place,  but  not  under  the  label  of  “CRM.”  (34) 

F-16  CRM  training  should  focus  on  single-seat,  not  multicrew,  aircraft. 

•  “Most  CRM  training  has  been  focused  on  crew  plans,  so  it’s  not  always  applicable,  but  the 
single-seat  guys  try  to  translate  it.”  (5) 

USAF  needs  to  show  that  it  is  serious  about  CRM. 

•  No  applicable  comments 
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Other  positive  comments 


•  Components  are  incorporated  in  daily  ops.  (11) 

•  “Briefing  Room  2000”  concept,  which  uses  PowerPoint  drawings  and  videos  to  emphasize 
key  points,  allows  the  flight  lead  to  give  a  more  comprehensive  briefing.  It  also  assures  that 
the  flight  lead  will  not  “draw  a  wrong  picture  on  board  of  radar,  HUD,  etc.  (22) 

•  For  two-seat  F-16  flights,  a  coordination  briefing  is  accomplished  which  covers  who’s  in 
charge,  and  stresses  situational  awareness  and  decision  making.  (22) 

•  As  a  flight  lead  you  cannot  do  everything  yourself,  so  you  have  to  delegate  responsibilities, 
like  radar  search,  radio  communications,  etc.  (32) 

•  CRM  helps  tell  pilots  what  might  go  wrong  before  it  do  something  wrong.  (1) 

•  CRM  directly  relates  “when  something  goes  wrong.”  (3) 

•  CRM  is  really  about  being  a  good  flight  leader/pilot.  (3) 

•  CRM  is  useful  and  effective  because  it  can  “keep  us  alive.”  (4) 

•  CRM  helps  you  fly  better  and  go  to  combat  better.  (11) 

•  CRM  is  using  everything  you  have,  to  do  the  best  you  can.  (3 1 ) 

•  In  CRM  you  should  use  the  wingman  in  the  briefing  and  encourage  them  to  speak  up  during 
the  flight.  (31) 

•  Special  interest  items  help  emphasize  potential  problem  areas.  (33) 

Other  negative  comments 

•  “CRM  training  is  all  about  ‘stating  the  obvious’.”  For  example,  they  tell  you:  “if  you  are 
headed  for  the  ground,  pull  up.”  (29) 

•  You  should  manage  your  cockpit  yourself;  you  don’t  need  help.  (29) 

•  CRM  issues  are  sometimes  addressed  because  of  the  training  environment,  but  there  is  too 
much  emphasis  on  the  syllabus  and  not  enough  flexibility.  (14) 

•  Easier  in  training  unit  than  operational  unit.  (18) 

•  In  F-16s  there  is  not  a  lot  of  opportunity  to  plan  together  because  of  the  tight  schedule,  but 
the  debriefs  are  good.  (13) 

•  You  can  pick  up  most  of  what  you  know  in  a  squadron,  regardless  of  the  academics  and  from 
the  IPs.  (7) 

•  Dedicated  CRM  courses  are  vague,  but  when  it’s  applied  is  when  the  different  components 
are  put  together  and  it  really  becomes  useful.  (10) 

•  “It  is  only  relevant  if  it  helps  meet  mission  objectives.”  (13) 

•  Sometimes  CRM  issues  are  discussed  in  reference  to  emergencies.  (30) 

Integration  of  CRM  into  everyday  operations  (Integration  into  simulator  missions! 

F-16  pilots  feel  that  they  are  already  doing  CRM, 

•  No  applicable  comments. 
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F-16  pilots  feel  that  they  are  being  “forced”  to  do  CRM. 


•  No  applicable  comments. 

Use  of  the  term  “CRM”  is  not  a  common  or  desired  point  of  reference  for  F-16  pilots. 

•  Sim  IPs  do  talk  about  some  CRM  behaviors  and  skills,  but  not  specifically,  and  never  with 
the  term  CRM.  (7) 

•  CRM  is  not  discussed,  except  for  things  like  communications  with  SOF,  etc,  during 
emergency  procedures.  (10)  (18) 

•  Mostly  procedural  in  handling  emergencies  and  carrying  out  to  logical  conclusion,  but  no 
mention  of  CRM  specifically.  (21) 

•  CRM  is  not  really  talked  about  in  the  simulator.  (35) 

•  Boeing  instructors  do  cover  some  CRM  in  the  simulator,  but  not  under  the  term  CRM.  (8) 

•  Boeing  IPs  are  good,  but  they  teach  the  “priorities  of  flight,”  not  CRM.  (13) 

F-16  CRM  training  should  focus  on  single-seat,  not  multicrew,  aircraft. 

•  Simulator  always  single-ship,  so  it  is  not  good  for  interflight  CRM.  (1) 

•  Simulators  are  focused  on  “single-seat”  training,  with  no  wingmen  or  leaders,  so  no  real 
CRM  taught.  (2) 

USAF  needs  to  show  that  it  is  serious  about  CRM. 

•  No  applicable  comments. 

Other  positive  comments 

•  Good  assessment  of  adhering  to  procedures  and  training  rules.  (1) 

•  Good  for  task  prioritization  (EPs,  etc.)  and  information  overload.  (1) 

•  Good  to  practice  “maintaining  aircraft  control”  during  EPs.  (1) 

•  Good  to  simulate  weather.  (1) 

•  If  student  asks  the  sim  instructor  to  be  the  SOF  or  to  read  the  checklist,  most  of  the  sim 
instructors  do  respond.  (35) 

•  During  simulator  checks,  the  respondent  (as  Stan/Eval)  plays  a  good  wingman  and  “reads  the 
checklist”  and  “looks  over  the  aircraft”  if  asked,  and  will  “give  radar  vectors”  as  ATC,  if 
asked.  (32) 

•  Boeing  instructors,  or  other  IPs,  will  read  checklist  or  do  SOF  duties  if  you  ask  them  to,  but 
not  much  discussion  of  CRM.  (34)  (33) 

•  Sometimes  the  stan  eval  flight  examiner  (SEFE)  giving  an  emergency  procedures  simulator 
will  tell  the  pilot  that  he  will  play  the  role  of  the  SOF,  the  wingman,  ATC,  etc.  The  SEFE 
will  read  the  checklist,  give  vectors,  obtain  the  weather,  or  whatever  is  requested  by  the  pilot 
in  the  simulator.  (36) 

•  Boeing  guys  are  very  professional  and  knowledgeable  in  the  simulator.  (22) 
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•  Some  prioritization  and  task  management.  (4) 

•  There  is  some  improvement  in  what’s  being  taught  in  the  sim;  the  contract  instructors  are 
experienced  fighter  pilots  and  do  a  good  job  explaining  “why”  you  need  to  do  something.  (6) 

•  The  decision  making  process  is  discussed.  (24) 

•  The  UTDs  (desktop  simulators  for  students  in  academic  building)  are  very  good  tools.  (25) 

•  Boeing  instructors  do  some  emergency  procedures  in  simulator.  (26) 

•  Although  Luke  IPs  do  not  give  the  students  simulators  except  the  air-to-air  TAC  ACES, 
when  this  pilot  “gets”  a  simulator,  he  calls  the  Boeing  instructor  to  “look  him  over”  as  the 
chase  pilot  and  also  uses  him  to  act  as  the  SOF  to  get  additional  information.  (31) 

•  B  Course  guys  see  all  new  things  in  the  simulator  first.  (22) 

•  Must  be  heavily  debriefed.  (1) 

Other  negative  comments 

•  CRM  is  occasionally  discouraged  in  the  simulator  because  some  of  the  simulator  IPs  don’t 
read  checklists  as  the  SOF,  as  it  is  taught  in  CRM  training.  (30) 

•  Not  much  emphasis.  (19) 

•  No  new  concepts  taught;  just  worked  on  task  prioritization.  (20) 

•  Boeing  guys  are  noncurrent  pilots,  so  that  creates  a  problem.  (29) 

•  Boeing  guys  do  not  have  time  to  look  at  things  like  CRM  because  they  are  working  the 
syllabus  requirements.  (29) 

•  Sometimes  negative  training  because  contract  instructor  give  procedures  which  are  different 
than  ones  currently  used.  (1) 

•  Not  being  given  in  simulator  because  simulators  are  tactically  oriented  for  specific  training 
missions.  (3) 

•  CRM  is  not  integrated  into  simulator  missions,  and  they  should  be.  (12) 

•  CRM  is  not  integrated  into  simulator  missions  because  the  pilots  are  too  busy  doing  “hands- 
on  stuff.”  (14) 

•  Use  TAC  ACES  simulator  for  air-to-air  set  ups,  but  no  CRM.  (26) 

•  CRM  is  not  done  intentionally  in  the  simulator.  (27) 

•  Emphasis  is  not  on  CRM,  but  rather  on  the  specific  mission  because  of  the  limited  time  in  the 
simulator.  (5) 

•  Nothing  specific,  but  some  of  the  individual  skills  (like  communications)  are  taught  in 
passing.  (9) 

•  CRM  subjects  are  covered  only  in  EP  debriefs.  (11) 

•  Squadron  IPs  rarely  instruct  in  the  simulator,  except  for  TAC  ACES.  (8) 

«_  - 

Instructors’  attitudes  of  students’  understanding  of  use  of  CRM 
F-16  pilots  feel  that  they  are  already  doing  CRM. 

•  Using  CRM  in  single-seat  fighters  should  already  be  inherent  to  the  students’  training. 
Everyone  already  emphasizes  to  call  the  SOF  if  you  have  any  problems.  (31) 
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F-16  pilots  feel  that  they  are  being  “forced”  to  do  CRM. 


•  No  applicable  comments. 

Use  of  the  term  “CRM”  is  not  a  common  or  desired  point  of  reference  for  F-16  pilots. 

•  IPs  do  not  ever  use  the  word  “CRM”,  but  do  emphasize  it.  (1 ) 

•  Word  CRM  creates  “hostile”  attitudes.  (1) 

•  There  is  “no  specific  talk  of  CRM,  but  students  are  willing  to  speak  up.”  (3) 

•  Students  understand  the  subtasks  of  CRM  but  do  not  like  the  term  “Crew  Resource 
Management”  because  they  say  that  they  are  “not  in  a  crew  airplane”  (4) 

•  They  do  not  understand  CRM  as  specific  training;  it  is  just  integrated  into  what  they  do.  (13) 

•  They  do  not  tie  CRM  principles  and  behaviors  to  the  term  “CRM.”  (23)  (29) 

•  Single-seat  pilots  hate  words  with  “crew”  in  them.  They  want  to  use  pilot  instead  of  crew, 
for  example:  pilot  van  vice  crew  van;  pilot  rest  vice  crew  rest;  and  pilot  meeting  vice  crew 
meeting.  (29) 

•  The  question  should  not  be  “do  they  understand  use  of  CRM,”  but  rather  “do  they  know  the 
buzzwords  for  CRM.”  (32) 

•  While  younger  pilots  understand  the  word  “CRM”  and  mention  it  occasionally,  older  pilots 
mention  the  topics,  but  not  the  word  CRM.  (34) 

F-16  CRM  training  should  focus  on  single-seat,  not  multicrew,  aircraft. 

•  No  applicable  comments. 

USAF  needs  to  show  that  it  is  serious  about  CRM. 

•  No  applicable  comments. 

Other  positive  comments 

•  There  has  been  more  emphasis  on  “roles  and  responsibilities”  lately.  (2) 

•  Positive  trend  over  the  last  5  years.  (3) 

•  TX  course  (short  transition)  students  have  no  problem  with  integrating  CRM  because  they 
are  experienced  and  have  seen  breakdowns  before.  (33) 

•  The  RTU  business  is  about  “showing  them  a  picture,  so  the  student  can  put  it  in  their  bag  of 
tricks.”  (1) 

•  Students  don’t  understand  the  importance  initially,  but  by  the  end  of  the  program  they  are 
beginning  to  learn  “how  to  time-share.”  (27) 

Other  negative,  comments 

•  Students  do  not  know  what  is  going  on  with  CRM  because  they  are  overloaded.  The  IPs  do 
the  CRM  for  them,  but  they  do  not  know  it.  (1) 
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•  “Students  come  into  F-16  formal  training  too  CRM  oriented  and  are  too  willing  to  speak  up 
right  away.  They  forget  that  they  are  in  a  single-seat  fighter.”  (12) 

•  “Students  are  struggling  with  CRM  because  they  are  being  taught  in  F-16  training  to  be  self¬ 
thinking  entities,  and  in  AETC  that  was  discouraged.”  (14) 

•  “They  understand  that  it’s  important  not  to  die  while  they’re  flying,  but  probably  not  more 
than  that.”  (24) 

•  One-third  have  the  big  picture  about  CRM  and  two-thirds  do  not  have  the  big  picture  and  get 
task  saturated  easily  because  of  the  upfront  controls  and  embedded  screens.  (26) 

•  B  course  (long)  students  do  not  understand  CRM  other  than  what  is  taught  in  class  because 
they  do  not  have  the  experience  to  know  what  might  break  down  in  flight.  (33) 

Students’  attitudes  of  instructors’  and  evaluators’  use  of  CRM 

F-16  pilots  feel  that  they  are  already  doing  CRM 

•  No  applicable  comments. 

F-16  pilots  feel  that  they  are  being  “forced”  to  do  CRM. 

•  IPs  have  always  done  CRM,  but  do  not  like  to  be  told  to  do  it.  (18) 

Use  of  the  term  “CRM”  is  not  a  common  or  desired  point  of  reference  for  F-16  pilots. 

•  Most  instructors  cover  some  CRM  issues,  but  they  do  not  call  it  CRM.  Issues  are  discussed 
while  covering  different  flying  principles.  (5)  (21)  (35) 

•  Most  IPs  use  parts  of  CRM,  but  never  refer  to  it  as  CRM.  (10) 

•  IPs  are  enthusiastic  about  flying,  but  never  mention  the  word  CRM.  (11) 

F-16  CRM  training  should  focus  on  single-seat,  not  multicrew,  aircraft. 

•  No  applicable  comments. 

USAF  needs  to  show  that  it  is  serious  about  CRM. 

•  No  applicable  comments. 

Other  positive  comments. 

•  Standard  communication  really  stressed.  (7) 

•  Situational  awareness  is  stressed  a  lot  and  is  vital  prior  to  entering  the  aircraft.  (7) 

Other  negative  comments. 

•  No  applicable  comments. 
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Suggestions  by  respondents  to  improve  CRM  training 


Make  all  CRM  training  tactically  relevant 

•  Use  tactical  scenarios  in  training  to  make  it  interesting  and  keep  students  minds  thinking 
about  it.  (1) 

•  Minimize  psychological  terms.  (1) 

•  Put  it  into  “fighter  pilot”  speak.  (1 )  (6) 

•  CRM  should  be  taught  in  every  segment  of  flight.  (25) 

•  Needs  to  oriented  to  the  real  world.  (28) 

•  Find  ways  to  put  CRM  topics  in  plain  English  and  assure  that  they  are  applicable  for  each 
specific  mission.  (34) 

•  War  stories  are  helpful.  Bar  talk  about  good  and  bad  things  that  have  happened  is  good.  (1) 

•  The  more  the  tactical  relevance  of  the  CRM  principles  can  be  applied,  the  better  the  outcome 
of  the  mission.  (2) 

•  Anytime  you  work  with  fighter  pilots  you  will  have  challenges.  Must  stress  “practicality” 
and  “relationship  to  mission  objectives.”  No  fluff!  (2) 

•  Must  emphasize  tactical  relevance  in  everything.  (23) 

Make  CRM  training  applicable  to  single-seat  fighters. ' 

•  Make  it  applicable  to  single-seat  fighter  pilots.  (1)  (2)  (5)  (23)  (33) 

•  Think  of  CRM  from  the  perspective  of  a  single-seat  pilot.  (12) 

•  The  effort  is  good,  but  CRM  needs  to  be  specialized  for  single-seat  aircraft  using  flight 
communication  techniques,  instead  of  using  multiplace  airplane  flight  deck  communication 
procedures.  (35)  (36) 

•  Show  application  of  CRM  to  single-seat  jet.  (23) 

Consider,  using,  another^  term  for  single-seat  fighters  other  than  Cockpit/Crew  Resource 

Management 

•  “CRM  is  a  dirty  word  to  the  fighter  community”,  don’t  ever  refer  to  CRM,  but  just  teach  the 
subtopics.  (4)  (6) 

•  Call  CRM  something  different  for  fighter  guys.  (4)  (6) 

•  Do  not  talk  “Crew  Coordination”,  talk  “Flight  Integrity.”  (6) 

•  Do  not  talk  “Aircrew  Member”,  or  Aircraft  Commander”  (6) 

•  Do  not  use  the  words  “CRM”  or  “Crew  Resource  Management;”  they  are  “dirty  words.”  Use 
only  subtopics.  (5)  (18) 

•  Change  the  name  CRM  to  something  that  is  not  labeled  as  being  from  the  heavy  community. 

(11) 

•  Make  CRM  training  more  transparent  to  the  pilots.  Discretely  put  it  into  other  information 
without  it  being  highlighted  as  a  separate  program.  (20) 

•  De-emphasize  multicrew  CRM  (the  genesis  of  CRM).  The  reaction  of  the  fighter  world  to 
CRM  is  to  “scoff’  at  it.  (23) 
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•  Must  “sell”  CRM  concepts,  but  not  the  word,  to  the  fighter  world  because  they  think  that 
they  are  already  doing  it.  (23) 

•  “Don’t  talk  about  CRM  per  se.”  (29) 

•  F-16  pilots  “shut  off’  at  the  use  of  the  term  “CRM.”  It  is  not  for  F-16  pilots  since  they  have 
no  crews.  (32) 

•  “F-16  pilots  do  not  use  management  or  resources,  and  the  word  ‘cockpit’  sounds  like  where 
you  put  things  in  the  cockpit,  so  just  drop  the  name  CRM.  Maybe  call  it  ‘fighter  pilot 
situational  awareness.”  (32) 

•  Do  not  use  the  word  CRM,  it  will  shut  single-seat  pilots  off.  (34) 

•  Change  the  term  from  “CRM.”  Do  not  use  buzzwords  that  send  the  wrong  message.  Maybe 
use  the  term  “Tactical  Resource  Management.  (36) 

•  Never  refer  to  CRM  as  "CRM"  because  it  will  turn  people  off.  (14) 

•  It  matters  what  you  call  the  training.  “Do  not  call  it  CRM.”  (18) 

•  The  name  CRM  probably  needs  to  be  changed  because  a  lot  of  single-seat  pilots  are  sensitive 
to  it.  (28) 

Assure  course,  delivery  is  applicable  to  audience 

•  Talk  about  instances  where  a  mission  was  messed  up  by  someone  not  listening  in  the 
briefing.  (12) 

•  Show  examples  of  things  that  have  gone  wrong  before  and  discuss  them.  (14) 

•  Use  of  case  studies  of  accidents  is  very  good.  (24) 

•  Classes  should  be  taught  visually  as  well  as  auditorially,  and  related  to  the  squadron’s 
mission.  (24) 

•  Single-seat  pilots  like  case  studies  on  where  things  have  broken  down  in  the  past  and  what 
could  have  been  done  to  prevent  incidents/accidents.  (26) 

•  Use  “role-playing”  by  giving  different  people  different  information  and  see  how  they 
coordinate  it  together.  (5) 

•  Do  not  use  old  videos  of  accidents  and  crashes  over  and  over  to  motivate.  (5)  (7) 

•  Use  recent  videos  of  good  and  bad  CRM.  (9)  (30) 

•  Have  more  “hands-on”  training.  (9) 

•  Have  more  “case-studies”  of  accidents  using  the  specific  aircraft  flown.  (9) 

•  ‘  Have  more  films  and  videos  and  then  talk  about  it.  (1 1) 

•  Case  studies  and  hands-on  experience  are  excellent.  (21) 

•  Prevent  complacency  by  using  examples  and  case  studies.  (21 ) 

•  Do  not  use  big-room  lectures  with  a  lot  of  “concepts.”  (21) 

•  “Sitting  in  a  room  and  talking  about  CRM  is  not  good.”  (13) 

•  Meld  CRM  with  safety  and  use  accidents  and  incidents  that  are  applicable.  (32) 

•  Go  over  accidents  and  where  the  flight  broke  down  (the  chain  of  events).  (33) 

•  Stress  how  actions  or  inaction  caused  problems  downstream.  (33) 

•  Having  a  discussion  on  safety  messages  is  a  good  format.  (13)  (19) 
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Assure  concept  of  training  (format,  duration,  frequency,  etc)  is  enhancing  to  learning 
process 

•  Do  only  quick  review  during  continuation  training.  (1) 

•  Shoot  straight.  (2) 

•  Keep  it  from  being  “Mickey  Mouse.”  (2) 

•  Give  it  in  a  time  of  day  when  students  can  absorb  material.  (3) 

•  Have  a  mixture  of  experience  in  the  class  to  keep  it  interesting.  (3) 

•  Stress  the  CRM  subtopics,  not  CRM  as  the  topic.  (4) 

•  Start  early  in  UPT  and  FT  and  continue  in  Flight  Lead  Training  and  IP  upgrades.  Instill 
CRM  before  the  “captains  and  majors  get  their  walls  up.”  (14) 

•  “Don’t  make  it  kinder  and  gentler,  touchy-feely  stuff.”  (14) 

•  Emphasize  “what”  we’re  teaching  instead  of  giving  it  the  new  name  of  CRM.  (25) 

•  Stress  how  to  “coordinate  a  four-ship”,  just  like  multicrew  aircraft  have  to  learn  how  to 
“coordinate  the  crewmembers  on  the  flight  deck.”  (27) 

•  Stress  how  to  set  things  up  in  the  cockpit  and  maximize  effectiveness.  (27) 

•  Since  most  of  CRM  is  common  sense,  bring  up  examples  of  things  that  did  not  work,  or 
things  that  worked  well.  (31) 

•  Keep  the  CRM  training  simple.  (3 1 ) 

•  Emphasize  “communication  and  briefings.”  (19) 

•  Stress  that  human  factors  are  important  in  a  risky  business  like  this.  (20) 

•  Do  CRM  training  more  often,  certainly  more  than  once  a  year.  (22) 

•  Needs  to  be  thought  provoking.  (28) 

•  Should  be  group  oriented.  (28) 

•  Should  be  every  six  months  for  an  hour  in  duration.  (28) 

•  Should  include  risk  management  and  ORM.  (28) 

•  “Concentrate  on  the  human  factors  side,  like,  “why  do  people  not  do  the  right  thing  like 
‘speak  up’  at  the  appropriate  time.”  (29) 

•  Make  the  objectives  broader,  so  as  not  to  be  individual  components.  (32) 

•  Don’t  get  wrapped  around  the  axle  with  the  components  of  CRM,  but  rather  focus  on  the 
underlying  issues.  (32) 

•  Need  to  stress  things  pilots  need  to  look  for  that  will  keep  them  out  of  trouble.  (33) 

•  Keep  CRM  training  short;  the  move  from  eight  hours  to  four  hours  was  good.  (33) 

•  It  is  useful  if  you  brief  “crunch  points,”  that  is,  points  where  things  may  go  wrong.  (34) 

•  Should  be  given  in  UPT  in  depth,  then  built-on  in  F-16  training.  (36) 

•  If  CRM  training  takes  too  long  of  a  time,  single-seat  fighter  pilots  will  go  to  it  with  a 
negative  attitude.  “The  shorter  you  can  make  continuation  training  (one  hour  desired),  the 
better.”  (31) 

•  CRM  training  is  useful  if  it  is  done  the  right  way  and  not  just  filling  a  square.  It  should  be 
more  frequent  and  not  just  annually.  (34) 

•  Must  stress  that  CRM  is  for  everything,  not  just  emergencies  and  contingencies.  (12) 
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Leaders  must  support  training 


•  Squadron  Commanders,  Operations  Officers,  and  Weapons  Officers  must  embrace  it.  (6) 

•  CRM  issues  should  be  covered  often  in  briefings.  (14) 

•  Present  it  as  part  of  normal  pilots’  meetings.  (24) 

•  It  should  be  emphasized  that  while  wingmen  need  to  talk  more,  the  flight  leads  need  to  stress 
in  briefing  that  wingmen  should  ask  questions  and  make  comments  if  appropriate.  (35) 

•  Energize  Squadron  Commander  and  Operations  Officers  to  set  the  pace  for  the  rest  of  the 
squadron.  (19) 

Use  simulators  to  reinforce  training 

•  Use  more  interactive  training  and  less  classroom.  Need  a  PC-based,  nonclassified, 
communications  interactive,  F-16  simulator  for  pilots  to  practice  mission  at  home.  (4) 

•  CRM  should  be  incorporated  into  emergency  simulators  to  see  how  the  pilot  uses  the  flight 
lead,  SOF,  etc.  (25) 

•  Consider  using  “eye-trackers”  in  the  simulator  to  stress  CRM  principles.  (27) 

•  Use  video  playback  of  simulator  or  training  device  missions  that  include  the 
“communication”  between  the  players.  (21) 

•  Change  EP  simulators  so  that  the  instructors  will  “play”  SOF,  etc.  (30) 

•  Use  the  Unit  Training  Devices  (UTDs)  more.  (13) 

•  Have  simulator  missions  specially  designed  for  CRM  with  multiple  roles  played  by  the 
simulator  instructor  off  of  a  script  stressing  interflight  and  external  communications  (SOF, 
etc).  (22) 

Do  not  “force”  training  to  be  accomplished 

•  “We  already  have  enough  ground  duties,  don’t  increase  them.”  (8) 

•  We  do  CRM  already,  we  don’t  need  to  be  trained  to  do  it.  (29) 

•  We  should  just  “do”  CRM,  not  “invoke”  that  we  do  it.  (1) 

•  Do  not  change  things  and  increase  the  workload  of  the  instructors  by  forcing  them  to 
formally  do  something  that  they  have  always  done.  (8) 

•  Don’t  make  CRM  training  something  else  “the  guys  have  to  do.”  (20) 

•  Integrate  what  is  already  being  done  in  CRM  without  increasing  the  total  time  required  of 
pilots  or  degrading  other  training.  (8) 

CRM  instructors  must  have  credibility 

•  Must  keep  credible  (fighter  experienced)  CRM  instructors.  (3) 

•  Delivery  of  CRM  training  is  very  important;  audience  must  relate  to  the  instructor  as  far  as 
background  and  knowledge.  (24) 

•  Do  not  have  the  aerospace  physiologist  teach  CRM  because  they  do  not  have  the  credibility 
to  F-16  pilots.  Have  F-16  pilots  teach  F-16  pilots.  (32) 
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General  comments. 


•  Keep  training  applicable.  Use  a  format  that  “takes  you  out  of  the  squadron  and  allows  you  to 
focus.”  (3) 

•  Stress  all  resources  to  help  pilots.  (19) 

•  Do  not  always  address  CRM  in  relationship  to  emergencies  (like  having  other  people  read 
checklists,  etc.  (12) 

•  Add  an  item  on  student  gradesheet  which  says:  “flight  lead/wingman  utilization”  to  help 
evaluate  EP  and  checklist  usage.  (25) 

•  CRM  seems  “detached  from  the  rest  of  the  training  syllabuses;  it  should  be  integrated.  (10) 

•  Recognize  that  the  F-15  community  has  even  more  single-seat  mentality  than  the  F-16.  They 
are  slow  to  change  from  the  old  mentality.  (18) 

•  Must  be  an  on-going  process  throughout  training.  (21) 

•  Discussing  mishaps  is  a  good  attention-getter,  but  you  have  to  realize  that  “hindsight  is 
always  20-20.”  (35) 

•  Stress  coordination  between  all  players.  (22) 

•  Consider  ops  doing  some  CRM  with  maintenance  personnel.  (33) 

•  Recognize  that  since  you  cannot  teach  maturity,  it  will  take  time  to  learn  the  applicability  of 
CRM.  (34) 

•  People  need  to  recognize  that  it  is  still  an  “individual  call”  by  the  flight  leads  on  whether  he 
will  allow  the  wingman  to  talk  on  the  radio.  (34) 

•  Since  CRM  deals  with  emergency  procedures,  it  should  be  discussed  during  the  “emergency 
of  the  day”  in  the  flight  briefing.  (26) 


Assessment  of  Learning  Styles  (n  =  35  assessed) 
Visual  (n  =  9.  25.7%) 

Respondents:  (12)  (14)  (10)  (30)  (35)  (19)  (23)  (29)  (34) 

Representative  Respondent  Remarks: 

•  Visual,  followed  up  by  flying.  (30) 

•  Must  see  it,  instead  of  hear  it.  (35) 

•  If  I  can  see  a  good  demo  or  a  picture,  I  can  recreate  it  later.  (34) 

Auditory  (n  =  0, 0%) 


Respondents:  0 


Hands-on  (n  =  17. 48.6%) 

Respondents:  (1)  (2)  (6)  (25)  (26)  (5)  (7)  (9)  (11)  (15)  (16)  (17)  (21)  (8)  (13)  (20)  (33) 
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Representative  Respondent  Remarks: 

•  I  have  to  read  a  little,  but  I  need  to  be  hands-on  to  really  learn.  (1) 

•  I  am  slow  to  learn  right  away,  but  when  I  touch  something,  my  learning  accelerates.  (2) 

•  I  learn  by  “doing  it.”  (6) 

•  Must  write  or  do  the  task  to  learn.  (26) 

Hands-on/Visual  (n  =  6, 17.1%) 

Respondents:  (3)  (4)  (31)  (18)  (22)  (36) 

Representative  Respondent  Remarks: 

•  Hands-on  for  flying,  visual  for  systems.  (1 8) 

•  Visual  learner  with  hands-on  a  close  second.  (22) 

Visual/Auditory  (n  =  3,  8.6%) 


Respondents:  (24)  (32)  (27) 

Representative  Respondent  Remarks: 

•  Must  do  both.  (27) 


No  Assessment  Made  (n  =  1) 


Respondents:  (28) 


APPENDIX  G 


CORE  CRM  BEHAVIORS  DATA  COLLECTION 
(Interviews  with  F-16  Instructor  Pilots,  Leaders  and  Students) 


Open  Coding  of  Data 

As  the  qualitative  data  were  collected,  reoccurring  themes  and  descriptive  words  were  used 
to  code  similar  topics  together  to  form  data  coding.  Coded  data  were  later  represented  in  various 
matrix  forms  to  aid  in  cross-referencing  the  responses.  Using  grounded  theory,  the  data  analysis 
included  open  coding,  which  involved  determining  what  themes  and  categories  emerged  from 
direct  responses  from  the  F-16  IPs,  unit  leaders,  and  F-16  students  (Creswell,  1994).  The  key 
phrases  were  collected  and  coded  into  meaningful  categories.  Although  all  key  phrases  were 
considered  for  use  in  the  narrative,  the  following  phrases  evolved  as  primary  coding  categories 
for  comparative  analysis.  This  data  has  not  been  axial  coded  into  further  reflective  topics. 


Opening  Coding  Topics 
Attitudes  on  Mission  Planning/Debrief 
Flight  Planning  and  Briefings 

•  Brief  a  good  plan  and  then  fly  as  briefed.  (9)  (25)  (29)  (31)  (36) 

•  Important  to  tailor  the  flight  to  the  lowest  common  denominator.  (1)  (4)  (20)  (24) 

•  IPs  need  to  read  the  grade  books  on  all  the  students  in  the  flight.  (1) 

•  “Squadron  standards”  are  taught  for  whom  is  responsible  for  which  duties.  (2)  (6)  (32)  (36) 

•  Everyone  must  get  involved  in  the  plan.  (2)(3)(3 1) 

•  Must  brief  task  prioritization:  visual  1st,  formation  2nd,  etc.  (4)  (6) 

•  Build  “smart  materials”  to  carry  into  the  cockpit  which  help  you,  but  don’t  overload  you. 
Then  use  those  materials  to  “chair-fly”  the  mission.  (6) 

•  Priorities  and  weather  are  covered.  (12) 

•  CRM  is  never  talked  about,  but  issues  are  covered.  (11)  (12)  (14)  (18)  (21)  (22) 

•  Briefing  is  directed  from  the  flight  lead.  (25)  (35) 

•  Incorporate  emergency  procedures  and  loss  of  visual  in  briefings.  (26) 

•  This  involves  where  you  put  stuff  in  the  cockpit.  (22)  (27) 

•  When  flying  air-to-air,  there  is  not  much  CRM  briefed,  but  in  air-to-ground,  there’s  more 
CRM  subjects  covered.  (5) 

•  Planning  is  taught  in  UPT,  but  not  much  by  IPs  in  formal  training.  (7) 

•  It  is  hard  to  label  what  is  CRM,  what  it  is  saying,  what  is  going  to  happen,  and  what  the 
priorities  are.  (10) 

•  Focus  mostly  on  route  study  and  relying  on  each  other.  (21) 

•  Not  much  covered  except  for  general  comments.  (30) 

•  We  do  what  we  need  to  be  successful.  (8) 
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•  Students  need  to  know  what  the  “priorities”  are.  (13) 

•  Delegate  responsibilities  so  that  the  students  do  most  of  the  planning.  (32) 

•  A  lot  of  assumptions  have  to  be  made,  like:  is  everyone  ready  to  fly,  do  they  have  the 
required  crew  rest,  etc.  (33) 

•  Does  not  follow  “squadron  standards”  for  flight  planning  responsibilities.  Because  of  student 
experience  levels,  normally  lets  people  volunteer  for  different  tasks.  (34) 

Debriefing 

•  CRM  issues  discussed  primarily  to  assess  what  went  wrong.  (2)  (3)  (10)  (1 1)  (12)  (21)  (31) 
(32) (33) (35) 

•  Need  to  focus  on  how  well  a  student  “executed”  the  objectives  of  the  mission.  (1)  (4) 

•  Need  to  assess  if  training  rules  were  followed.  (1)  (6) 

•  Need  to  assess  communication,  situational  awareness,  task  saturation/management.  (1) 

•  It  should  be  emphasized  that  there  is  “no  rank”  in  a  debriefing.  (1) 

•  Safety  aspects  should  always  be  emphasized.  (1) 

•  Place  where  CRM  comes  into  play:  “If  they’re  not  managing  their  cockpit,  they  bust  the 
ride.”  (6) 

•  Assess  if  they  always  had  “the  big  picture.”  (6) 

•  Assess  if  they  were  prioritizing  correctly.  (6) 

•  Debriefing  is  more  of  a  “discussion”  of  what  was  done.  (25) 

•  Debriefing  involves  discussion  of  techniques  to  improve.  (27) 

•  In  debrief,  the  entire  tape  is  never  reviewed  because  of  lack  of  time;  accordingly,  some  CRM 
behaviors  might  be  missed.  (18) 

•  Debriefings  are  good  from  a  review  of  the  flight  perspective.  (13) 

•  CRM  in  debrief  is  not  a  large  factor;  you  only  assess  what  happened  -  good  or  bad.  (20) 

•  During  the  debriefing,  the  outcomes  and  lessons  learned  are  covered  to  make  ourselves 
better.  Sometimes  the  squadron  weapons  officer  disseminates  the  lessons  learned.  (36) 


Attitudes  on  Crew  (Flight)  Coordination/Flight  Integrity 
Roles  of  a  four-ship  flight  leader 


•  Primary  leader  and  decision-maker  for  the  flight.  (2)  (3)  (5)  (7)  (1 1)  (14)  (21)  (22)  (23)  (24) 
(25)  (29)  (30)  (34)  (35) 

•  Responsible  to  get  the  mission  accomplished  and  keep  everyone  safe.  (1)  (8)  (12)  (19)  (20) 
(24)  (27)  (29)  (30)  (31)  (32)  (33)  (36) 

•  Lead  must  be  ready  to  implement  contingencies.  (5)  (7)  (12)  (14)  (18)  (22)  (33) 

•  Conduct  a  good  brief  and  debrief.  ( 1 )  (8)  ( 1 0)  ( 1 2)  (26) 

•  Encourage  flight  members  to  speak  up.  (1)  (23)  (25)  (26)  (35) 

•  Maintains  situational  awareness  for  the  flight  (1)  (9)  (14)  (18)  (22)  (31)  (36) 

•  Know  flight’s  capability  and  adjust  accordingly.  (9)  (12)  (13)  (31)  (36) 

•  Provides  “integrity”  for  flight.  (2) 

•  Makes  sure  everyone  knows  their  “role”  in  the  four-ship.  (3) 
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•  Lead  “supports  the  wingmen”  when  needed  by  reading  the  checklist,  flying  wing,  and 
making  recommendations.  (4) 

•  Four-ship  must  be  integrated.  “Without  integration,  we  lose  wars.”  (6) 

•  Keep  plan  simple.  (6) 

•  Know  the  standards.  (6) 

•  Employ  effectively  and  achieve  briefed  goals.  (27) 

•  Flight  lead  has  the  hammer  for  the  flight,  but  anyone  can  knock-it-off.  (25) 

•  Stress  good  communication.  (26) 

•  “Needs  to  fly  the  plane  without  thinking  and  think  about  others  in  the  flight.”  (5) 

•  Competent  pilot  himself.  ( 1 0) 

•  Has  the  experience  to  be  an  example  to  the  rest  of  the  flight.  (18) 

•  Administrative  functions.  (23) 


Roles  of  an  element  leader 

•  Responsible  for  number  four.  (1)  (3)  (9)  (10)  (11)  (19)  (22)  (23)  (27)  (30)  (33)  (34)  (36) 

•  Maintain  integrity  with  number  four.  (31) 

.  Backup  for  lead.  (1)  (7)  (14)  (18)  (19)  (21)  (22)  (23)  (24)  (26)  (27)  (29)  (31)  (35)  (36) 

•  Ready  to  take  over  the  flight  if  lead  drops  out.  (2)  (3)  (4)  (12)  (14)  (36) 

•  Pickup  secondary  roles  as  directed  by  lead  or  as  a  split-off  element.  (4) 

•  Critical  for  flight  integrity.  (6) 

•  Same  as  lead,  but  he  is  expected  to  speak  up  if  he  sees  anything  going  wrong.  (25) 

•  Interact  with  the  flight  leader.  (31) 

•  As  tasked  by  lead.  (9)  (18) 

•  Works  with  lead  as  a  team.  (10) 

•  Good  follower.  (10) 

•  No  decision  authority  in  the  flight,  just  advisory.  (30) 

•  Ensure  team  efficiency  to  obtain  objectives.  (8) 

•  Help  meet  objectives  as  established  by  lead.  (33) 

•  Expected  to  have  more  situational  awareness  than  wingmen.  (8) 

•  Provide  support  to  lead  during  emergencies,  aircraft  malfunctions,  and  decision  making.  (8) 

•  Fly  his  own  jet  first,  then  backup  lead.  (13) 

•  “He  had  better  support  lead’s  game  plan  or  lead  should  ‘rip  his  head  off.”  (20) 

•  Alternate  flight  leader  and  decision  maker  to  safeguard  the  leader.  (22)  (23) 

•  Telling  lead  if  he  is  doing  something  wrong  or  is  missing  something.  (29)  (32) 

•  Clarifying  the  situational  awareness  (“the  picture”)  if  needed.  (29) 

•  Tries  to  read  lead’s  mind  and  support  what  he  wants.  (34) 

•  Support  the  game  plan.  (36) 

Roles  of  a  wingman 


•  Maintain  good  situational  awareness.  (1)  (7)  (8)  (10)  (12)  (22)  (25)  (31) 

•  Stay  visual.  (1)  (14)  (19)  (22)  (24)  (29)  (32)  (34) 

•  Stay  in  position.  (5)  (7)  (9)  (19)  (22)  (24)  (29)  (30)  (34)  (35) 
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•  Know  the  plan  and  fly  as  briefed.  (3)  (10)  (11)  (12)  (14)  (20)  (21)  (23)  (26)  (27)  (35)  (36) 

•  Be  willing  to  speak  up  when  necessary.  (1)  (3)  (4)  (7)  (10)  (12)  (18)  (27)  (29)  (33)  (35)  (36) 

•  Support  the  flight  lead.  (2)  (24) 

•  Know  how  to  “be  part  of  the  team.”  (6) 

•  Know  what  the  flight  lead  wants  and  how  to  contribute  to  the  flight.  (6). 

•  Sound  smart  on  the  radios.  (24) 

•  Put  bombs  on  target.  (24) 

•  Try  to  stay  two  steps  ahead  of  where  the  flight  leader  is.  (25) 

•  Be  safe.  (3 1 ) 

•  Do  basic  mission  planning.  (31) 

•  Know  what  the  plan  is  and  the  thought  process  for  the  plan.  (5) 

•  Know  what  the  flight  lead  wants.  (5) 

•  “QC  what  the  flight  lead  is  doing.  (5) 

•  Be  on  the  radio  frequency.  (9) 

•  Be  ready  to  take  over  lead  if  necessary.  (9) 

•  Be  able  to  adapt  to  situation.  (9) 

•  Tries  to  learn  from  watching  the  flight  lead.  (10) 

•  Look  out  for  anyone  who  is  task  saturated.  (10) 

•  Know  your  place,  but  do  not  let  something  go  by  that  you  could  have  prevented.  (21) 

•  No  one  is  exempt  from  a  bad  day,  so  the  wingman  must  be  able  to  do  something  if  lead  has 
problems.  (21) 

•  Keep  a  good  visual  search.  (30) 

•  Provide  mutual  support  for  rest  of  flight.  (30) 

•  Be  mentally  prepared  and  check  prerequisites.  ( 1 9) 

•  Be  ready  to  fly,  mentally  and  physically.  (19) 

•  Be  involved  in  mission  planning  and  prepared  for  his  part  of  flight  briefing.  (19) 

•  Be  ready  for  the  debrief  and  tape  rewound.  (19) 

•  Speak  by  exception.  (23) 

•  Assist  with  emergency  chase.  (23) 

•  Do  as  he’s  told.  (23) 

•  Make  sure  that  the  leader  does  not  get  killed  from  air-to-air  threats  or  ground-to-air  threats. 
(32) 

•  It  depends  on  experience  level.  Experienced  wingmen  can  do  more  than  new  wingmen.  (33) 

•  Assist  in  accomplishing  the  mission  objectives.  (33) 

•  Do  not  hit  anyone.  (34) 

•  Support  lead/three.  (34) 

•  Do  not  second-guess  lead/three.  (34) 

At  what  point  does  a  wingman  question  or  nonconcur  with  a  flight  or  element  leader’s 
action  or  decision 


•  If  safety  of  flight  dictates.  (1)  (2)  (3)  (5)  (8)  (9)  (10)  (11)  (12)  (13)  (14)  (21)  (22)  (23)  (25) 
(26)  (29)  (32)  (34)  (35)  (36) 

•  If  comfort  level  is  exceeded.  (1)  (25)  (31)  (35) 


70 


•  If  there  is  something  important  that  needs  to  be  brought  to  the  attention  of  the  flight.  (26) 

•  If  fuel  becomes  an  issue.  (25) 

•  If  rules  (training  or  ROE)  are  violated,  or  are  about  to  be  violated.  (14) 

•  To  clarify  a  situation  or  build  situational  awareness.  (1)  (36) 

•  "  Wingman  is  responsible  for  his  jet,  he  cannot  let  the  flight  lead  put  him  in  comer.”  (3) 

•  When  information  needs  to  be  passed  to  lead  to  build  situational  awareness  and  when 
communicating  a  suggestion,  but  never  any  direction.  (4) 

•  Some  pilots  still  in  old  school:  wingman  keeps  quiet  and  does  not  garbage-up  the  radios.  (6) 
(26) 

•  Most  pilots  lean  toward  new  school:  Let  the  wingman  speak  up  and  build  experience  to 
know  when  to  talk.  Some  people  use  the  term  “free-call,”  meaning:  if  you  don’t  like 
something  or  want  to  highlight  something,  you  call  it  out.  (6)  (26) 

•  If  it  looks  like  a  life-or-death  situation,  wingmen  should  speak  out.  However,  if  it  is  not 
appropriate,  think  about  it,  and  talk  about  it  in  the  debriefing.  (6)  (24) 

•  If  unclear  on  what  is  supposed  to  be  done.  (14) 

•  If  something  is  different  that  what  was  expected,  asks  for  clarification.  (27) 

•  This  is  based  on  the  experience  level  of  the  wingman,  as  to  whether  they  can  “know”  that 
something  is  going  wrong.  (3 1) 

•  “Sometimes,  flight  leads  brief  for  wingmen  not  to  say  anything  except  ‘two.’”  (5) 

•  When  the  wingman  can  offer  some  missing  information.  (5) 

•  If  it  is  not  a  dangerous  situation,  make  a  note  and  discuss  it  in  debrief.  (9)  (10)  (1 1)  (21) 

•  Before  you  say  or  do  anything,  make  sure  you  have  all  the  factors.  (10) 

•  Try  to  be  trusting  of  your  leader,  but  save  your  own  jet  at  all  costs.  (10) 

•  Rank  should  not  be  a  factor.  (21) 

•  Never  seen  that  situation.  (18) 

•  It  does  not  matter  how  you  say  it,  diplomacy  should  not  be  a  factor.  (2 1 ) 

•  “If  you  see  anything  that  is  dumb,  dangerous,  or  different.”  (30) 

•  It  is  very  hard  to  do.  Told  from  the  very  beginning  in  Introductory  Fighter  Fundamentals  that 
the  flight  lead  is  “God”  and  whatever  he  says  goes.  (30) 

•  Unbriefed  actions,  which  are  not  safety  of  flight,  should  be  “noted”  and  covered  in  the 
debriefing.  (8) 

•  If  a  wingman  sees  something  that  needs  to  be  communicated  for  situational  awareness  (e.g., 
bandits  not  previously  called  out).  (8)  (20)  (29) 

•  Fill  in  the  blanks  in  the  flight.  (20) 

•  Be  active  participant,  don’t  be  a  “water  skier.”  (20) 

•  Be  “directive”  for  anything  that  relates  to  wingman’s  own  plane.  (8) 

•  Lead  must  “honor”  a  wingman’s  call,  but  then  talk  about  it  in  debrief  as  to  whether  it  was  a 
right  or  wrong  call.  (13) 

•  When  task  saturated  and  have  a  problem  keeping  up,  speak  out.  (19) 

•  “Error  on  saying  too  much,”  instead  of  too  little.  (19) 

•  Can’t  let  an  overpowering  leader  keep  you  from  saying  something.  (19) 

•  Speak  up  in  the  briefing  if  something  is  wrong.  (33) 

•  In  the  air,  speak  up  immediately  if  something  does  not  look  right.  (33) 
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•  If  the  leader  does  not  concur  with  the  wingman’s  comment,  he  owes  the  wingman  a  response 
so  he  knows  what  is  going  on.  (33) 

•  If  they  do  not  like  the  “flow”  of  what’s  happening,  that  is,  it  is  dumb,  different  than  briefed, 
or  dangerous,  speak  out.  (34) 

•  A  wingmen  has  to  trust  the  lead  up  to  a  point.  (36) 

How  does  a  wingman  question  or  nonconcur  with  a  flight  or  element  leader’s  action  or 

decision  (what  does  the  wingman  “say”! 

•  Be  directive:  “Terminate”  or  “knock  it  off.”  (1)  (7)  (12)  (13)  (14)  (18)  (31)  (32)  (35)  (36) 

•  Ask  a  question.  (2)  (9)  ( 1 1 )  ( 1 2)  ( 1 4)  (24)  (29)  (3 6) 

•  Make  a  statement  in  plain  English.  (2)  (4)  (5)  (7)  (9)  (19)  (20)  (21)  (22)  (23)  (25)  (27)  (29) 

(31)  (30)  (33)  (34)  (35)  .  ' 

•  Diplomatically  hint  to  lead:  “two  is  4000  lbs”  instead  of  “lead,  you  forgot  the  ops  check,”  or 
“alpha  check  to  bullseye”  instead  of  “lead,  you  just  went  out  of  the  area.”  (24) 

•  “Lead,  I’m  not  comfortable  with....”  (25)  (27) 

•  Leading  Comment:  “Lead,  check  altitude....”  (26) 

•  “Free-calls”  from  anyone  in  flight,  like  “cameras  on”.  (26) 

•  Suggestion:  “Lead,  suggest  we  hook  left.... (3 1) 

•  Comment  to  gain  attention:  “Lead,  Two  shows  boundary  three  miles  north.”  (31) 

•  Ask  for  “clarification”  from  lead.  (10) 

•  The  tone  of  what  is  said  is  more  important  than  the  words  used.  (35) 

•  Request:  “Lead,  request  that . ”  (34) 

•  Statement:  “Lead,  that  is  unbriefed....”.  (34) 

•  If  not  critical,  find  the  right  “time”  to  ask  the  question.  (36) 

Attitudes  on  Risk  Management/Decision  Making 

How  does  a  flight  leader  assess  risks  in  a  four-ship 

•  Consider  experience  and  ability  of  flight  members.  (2)  (8)  (13)  (14)  (20)  (24)  (25)  (26)  (29) 
(30)  (31)  (33)  (34) 

•  Starts  on  the  ground.  (2)  (7)  (9)  (10)  (12)  (14)  (18)  (21)  (23)  (27)  (31) 

•  Consider  weather:  it  may  be  legal,  but  on  the  edge.  “Ask  if  everyone  in  flight  is 
comfortable.”  (1) 

•  Sometimes  the  flight  lead  must  call  a  “knock-it-off,”  even  if  it  is  legal.  Some  pilots  may  not 
say  anything  even  if  they  are  uncomfortable  because  they  do  not  want  to  be  thought  of  as  a 
wimp.  (1) 

•  Flight  leads  must  emphasize  that  it  is  ok  for  flight  members  to  say  that  they  are  not 
comfortable.  (1) 

•  Number  1  is  in  charge  and  makes  the  call,  but  number  three  has  a  vote.  Number  one  should 
ask  #3  if  he  has  any  doubts.  (3). 

•  In  “ORM”  you  think  about  a  “risk  assessment  chart”  and  going  through  it.  You  can  do  some 
of  this  in  mission  planning,  but  you  cannot  do  this  during  actual  flight  operations.  (6) 
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•  Knowing  “what  we  are  going  to  do,  and  can  we  do  it.”  (12) 

•  Problems  like  communications,  situational  awareness,  etc.,  might  require  a  reduction  in  the 
difficulty  of  the  mission.  (12) 

•  Need  to  determine  how  long  you  can  let  something  go  to  learn  something,  in  contrast  to 
letting  it  go  and  waste  the  gas.  (26) 

•  Listen  to  how  wingman  sounds  on  the  radio;  if  he’s  missing  radio  calls,  he  is  probably  task 
saturated.  (27) 

•  Keep  the  big  picture  to  make  decisions.  (5)  (10) 

•  Stay  away  from  air-to-ground  threats.  (7) 

•  He  considers  the  weather,  mission  objectives,  and  syllabus.  (1 1)  (35) 

•  No  formal  “numerical  risk  assessment”  like  in  ORM.  (11) 

•  Risks  should  be  assessed  based  on  the  situation.  (21) 

•  Consider  the  mission  (wartime  or  training).  (30) 

•  Consider  the  environment  (weather,  etc.).  (30) 

•  The  training  rules  are  important  to  most  risk  assessment  situations.  (35) 

•  Look  far  enough  forward  to  assess  risks  and  avoid  a  “square  corner.”  (8) 

•  Listen  to  radio  to  hear  if  someone  is  overtasked.  (13) 

•  Listen  to  radio  to  hear  if  wingman  is  not  responding  to  calls,  indicating  that  they  are  task 
saturated.  (29) 

•  Watch  formation  to  see  if  everyone  is  flying  the  game  plan.  (13) 

•  Flight  leader  should  use  all  of  his  resources  (element  lead,  SOF,  Top-3).  (19) 

•  “Identify  whether  it’s  a  risk  or  just  the  cost  of  doing  business.”  (32) 

•  Look  at  all  of  your  options  and  then  make  the  decision.  (32) 

•  Ask  for  suggestions  from  other  flight  members.  (34) 

How  does  a  wingman  assess  risks  and  make  decisions 

•  Know  your  own  limitations  and  whether  you  can  handle  the  task;  if  not,  speak  up.  (1)  (24) 

•  Stay  focused  on  “personal  limitations.”  (2) 

•  Prioritize  actions  and  stop  when  risk  is  too  great.  (2)  (4) 

•  This  is  very  “situational”  in  a  single-seat  fighter.  You  must  just  weigh  the  risk  of  what  is 
actually  happening  at  the  time.  (6) 

•  It  is  based  on  personal  comfort  level.  (25) 

•  Watch  your  own  capabilities  and  be  willing  to  “knock-it-off  ’  if  needed.  (31) 

•  Wingman  will  be  less  assertive  because  he  may  not  have  the  big  picture.  (5) 

•  Know  lead’s  game  plan.  (7) 

•  Keep  big  picture.  (10) 

•  Make  sure  you  feel  good  physically.  (30) 

•  Do  your  G-awareness  turns.  (30) 

•  Make  sure  you  are  emotionally  fit  and  have  your  “head-in-the-game.”  (30) 

•  Stay  ahead  of  the  aircraft  and  do  not  overextend  capabilities.  (8) 

•  Listen  to  radios  for  important  information.  (13) 

•  The  wingman  needs  to  trust  the  flight  lead,  but  only  so  far.  Then  he  must  take  responsibility 
for  his  own  action.  (32) 
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At  what  point  does  a  flight  leader  ask  for  recommendations/  comments”  from  outside  the 
flight? 


•  Keep  it  within  the  flight  whenever  possible.  (3)  (9)  (18)  (20)  (21) 

•  When  things  don’t  go  as  planned.  (4)  (8)  (22)  (23)  (29) 

•  Contact  the  SOF  for  emergencies.  (9)  (10)  (26)  (27)  (35) 

•  Contact  the  top  three  for  additional  info  (rules,  weather,  maintenance,  procedures).  (3)  (8) 

(11)  (18)  (22)  (23)  (27)  (29)  (35) 

•  If  they  are  confused  or  messed  up  (like  question  on  training  rules),  they  should  ask  another 
member  of  the  flight.  (1) 

•  When  they  are  overwhelmed  or  stumped.  (1)  (4) 

•  “People  are  more  comfortable  now  with  asking  questions  than  in  the  past.”  (1) 

•  “As  soon  as  he  needs  it.  (2). 

•  “If  there  is  any  risk  (if  I  have  to  think  for  more  than  5  seconds)  I  will  ask  a  question”  (SOF 
for  weather  and  alternates/  Top-Three  for  procedures  and  rules.  (3) 

•  When  they  need  more  information.  (4) 

•  People  don’t  want  to  “look  stupid”  and  would  rather  do  it  on  their  own  because  of  their 
fighter  pilot  mentality.  (6) 

•  Need  more  humility  in  flying  and  “fess-up”  more.  (6) 

•  If  a  flight  lead  ask  for  recommendations  in  front  of  students  or  wingmen,  it  “sets  the 
example.”  (6) 

•  Should  talk  to  Top-Three  prior  to  the  briefing  to  see  he  has  anything  important  to  consider. 

(12)  (14) 

•  In  the  air,  never  ask  advise  from  the  SOF  or  Top-Three,  but  listen  to  their  information  for 
your  own  decision.  (12)  (25) 

•  Go  to  the  SOF  anytime  there’s  a  need  for  “a  relaxed,  informed  recommendation.”  (24) 

•  Go  through  the  checklist  with  the  SOF.  (26) 

•  Never  ask  the  SOF  for  a  recommendation.  Only  tell  him  what  your  are  going  to  do  and  then 
let  him  suggest  a  different  approach  if  he  has  more  information.  But  the  flight  lead  in  the  air 
makes  the  final  decision.  (31) 

•  Only  if  there  is  time.  (5) 

•  Did  not  know  of  a  situation  where  the  flight  lead  would  ask  for  information  or  a 
recommendation,  but  realized  that  there  might  be.  (7) 

•  Mentality  today  is  that  “it’s  a  one-mistake  Air  Force,”  so  check  with  someone  else  who  is  not 
flying  like  the  SOF  or  Top-Three.  (21) 

•  Flight  leads  are  not  invincible,  they  should  check  on  something  if  they  have  a  question.  (30) 

•  If  there  are  sources  who  have  more  or  better  information  than  the  flight  lead,  the  flight  lead 
should  use  them.  Those  sources  are  at  zero  knots  and  1-G,  so  they  are  in  a  good  position  to 
do  the  research  and  make  recommendations.  (35) 

•  Talking  to  SOF  and  Top-Three  is  for  advisory  only,  the  flight  lead  makes  all  decisions.  (13) 

•  Whenever  it  is  needed  (weather,  importance  of  training,  etc.).  (19) 

•  “There  is  too  much  supervision  micromanagement  going  on.  Too  many  guys  call  the  SOF  or 
Top-Three.  We  push  guys  too  far  to  get  others  to  make  their  decisions  for  them.”  (20) 

•  Number  one  should  pass  number  four’s  problems  to  number  three  to  work/monitor.  (29) 

•  Anytime  the  mission  cannot  be  conducted  in  a  safe  manner.  (32) 
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•  Anytime  the  flight  lead  can  get  “real-time  information,”  he  should  go  to  that  source.  (32) 

•  If  he  needs  clarification  on  a  rule  or  regulation.  (33) 

•  Ask  someone  if  you  have  a  question  and  don’t  know  the  answer:  “knowledge  is  power.”  (33) 

•  A  day  or  two  before  the  flight  when  the  names  go  up  on  the  scheduling  board.  Check  flight 
members’  experience  and  currencies  and  get  the  latest  tactics  for  the  mission.  (34) 


Attitudes  on  Task  Management 

How  does  a  flight  leader  distribute  tasks  and  workload  within  a  four-ship? 

•  Start  with  mission  planning/briefing.  (1)  (4)  (5)  (8)  (10)  (1 1)  (12)  (19)  (20)  (21)  (22)  (23) 

(29)  (33)  (34)  (35) 

•  Know  abilities  of  flight  and  distribute  accordingly.  (2)  (6)  (7)  (19) 

•  Use  squadron  standards.  (3)  (4)  (8)  (9)  (19)  (20)  (22)  (23)  (25)  (26)  (27)  (31)  (33) 

•  Use  element  leader.  (3)  (14)  (25)  (26)  (27)  (32)  (33)  (34) 

•  If  you  have  to  change  duties,  brief  it  on  victor  (inter-flight  radio).  (1) 

•  Give  a  new  game  plan  if  things  are  not  going  as  briefed.  (1)  (12) 

•  In  the  air,  brief  a  new,  simple  plan  if  things  are  not  going  as  briefed.  (31) 

•  Make  changes  as  early  as  possible  to  give  people  a  chance  to  think  about  it.  (1) 

•  In  the  air,  risk  assessment  must  be  made  by  the  most  experienced  pilot,  and  the  wingmen 
must  trust  their  lead  unless  they  have  more  information.  (4) 

•  In  air-to-air  engagement,  Acceptable  Merge  Risk  (AMR)  must  be  predetermined  and 
considered.  (4) 

•  “More  critical  in  an  operational  unit  than  a  training  unit.  (6) 

•  In  the  air  things  change  and  flight  lead  may  distribute  tasks.  (9) 

•  Minimize  changes  in  flight.  (10) 

•  “Flight  lead  can  call  an  audible  in  the  air  but  it  must  be  acknowledged.”  (1 1) 

•  Pass  off  additional  or  new  tasks  and  duties  on  the  radio.  (35) 

•  Establish  radar  sort  responsibilities.  (18) 

•  Renumbering  the  flight  requires  a  “briefed”  reshuffling  of  responsibilities.  (18) 

•  Give  specific  tasks  to  the  wingmen  (like  sorting  targets)  to  decrease  the  flight  lead’s 
workload.  (30) 

•  Expect  wingmen  to  ask  in  the  briefing  if  there  are  any  questions  about  their  responsibilities  in 
the  flight.  (8) 

•  Stick  to  the  standard  tasks  game  plan  as  briefed,  but  when  calling  an  audible,  you  can  send 
wingmen  to  check  weather,  etc.  (13) 

•  Do  “visualization”  in  your  mind  while  looking  outside.  (19) 

•  “Do  not  keep  secrets;”  let  everyone  in  flight  know  what  is  happening.  (20) 

•  Give  wingmen  as  many  “black-and-white”  decisions  as  possible.  (20) 

•  Monitor  the  wingmen  to  make  sure  that  they  are  not  getting  task  saturated.  (22) 

•  Delegate  tasks  in  flight  (such  as  during  a  divert,  distribute  responsibilities  like 
communication,  alternates,  ranges,  gas,  etc.).  (29) 

•  It  depends  on  the  complexity  of  the  task.  Number  3  gets  the  complex  tasks,  and  2  and  4  get 
the  easy  tasks.  (32) 
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•  In  the  air,  if  he  can  do  it  himself,  he  should.  The  next  person  who  should  be  tasked  is 
Number  3  (to  change  frequencies  and  check  on  things,  etc.).  Three  can  delegate  some  of  the 

.  tasks  to  4.  (33) 

•  If  a  two-ship,  keep  Number  2  on  the  frequency  and  Number  1  jumps  around  the  frequencies 
to  gather  information.  (33) 

•  Leads  should  listen  for  missed  radio  channel  changes  and  fuel  checks  to  determine  if 
someone  is  overtasked.  (33) 

•  Do  not  hit  the  rocks,  other  airplanes,  etc.  (34) 

How  does  an  individual  pilot  handle  task  and  workload  management? 

•  Prioritize  (aviate,  navigate,  communicate).  (1)  (2)  (3)  (5)  (9)  (10)  (12)  (13)  (14)  (18)  (24) 

(25)  (27)  (29)  (31)  (33)  (35) 

•  Prepare  in  advance.  (5)  (7)  (10)  (19)  (29) 

•  Know  your  limitations.  (2)  (14)  (20) 

•  Slow  things  down  and  reduce  tasks.  (1)  (33)  (35) 

•  Organize.  (19)  (21)  (34) 

•  Know  the  “clues”  that  you  are  approaching  the  edge  of  your  envelope  and  how  to 
communicate  that  you  are  “unable”  to  do  that  particular  tasking.  (2) 

•  Must  quickly  learn  the  best  way  to  do  “cockpit  tasks.”  (4) 

•  Do  not  let  a  wingman  or  a  leader  drag  you  into  something  that  is  going  to  get  everyone  in 
trouble.  (14) 

•  Missing  radio  calls  is  the  first  cue  of  pilot  overload.  (24) 

•  Remember  responsibilities:  “Don’t  hit  the  ground  and  stay  in  formation  so  you  don’t  cause 
any  problems  for  lead.”  (26) 

•  Ask  lead  if  there  is  every  a  question.  (27) 

•  “Do  the  best  I  can.”  (7) 

•  Focus  on  flying  the  jet.  (9)  (18)  (35) 

•  Say  “unable”  on  radio  if  you  are  overloaded.  (9) 

•  “I  talk  to  myself  a  lot  to  try  to  remind  myself  what’s  next  and  what  my  priorities  are.”  (11) 

•  Do  the  emergency  procedure  “CAPs”  as  soon  as  you  can.  (18) 

•  Don’t  concentrate  on  any  one  thing  too  long.  (30) 

•  Keep  your  eyes  moving.  (30) 

•  Use  “reminders”  (like  turning  on  the  HUD  reticle)  to  remind  you  about  switches,  etc.  (30) 

•  Do  not  hit  ground  or  anything  attached  to  it  and  don’t  hit  each  other.  (8)  (23)  (29) 

•  “Fly  own  airplane  first.”  (13)  (33) 

•  Support  lead’s  game  plan.  (20) 

•  Realize  when  you  are  getting  behind,  are  task  saturated,  and  have  channelized  attention.  (22) 

•  Consider  yourself  in  the  worst  situation  and  think  about  what  you  might  do.  (29) 

•  Use  acronyms  for  memory  joggers  and  pacing  to  accomplish  the  correct  task  at  the  correct 
time.  (32) 

•  Be  able  to  recognize  task  saturation,  channelized  attention,  and  complacency,  and  figure  out 
where  you  are  most  likely  to  encounter  them.  (34) 
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Attitudes  on  Communication 


Types  of  factors  that  cause  a  breakdown  in  interflight  communication 

•  Not  speaking  up  when  necessary.  (1)  (8)  (9)  (11)  (12)  (21)  (26)  (31)  (35) 

•  False  assumptions.  (1)  (2)  (7)  (3 1)  (35) 

•  Personality  issues.  (2)  (3)  (5)  (6)  (14)  (19) 

•  Individual  task  saturation.  (4)  (1 1)  (23)  (25)  (34) 

•  Poor  radio  discipline.  (5)  (6)  (9)  (10)  (13)  (14)  (20)  (33) 

•  Equipment  problems.  (4)  (20)  (27)  (31)  (36) 

•  Improper  use  of  terminology.  (6)  (10)  (12)  (13)  (14)  (18)  (20)  (22)  (23)  (26)  (30)  (34) 

•  When  things  don’t  go  as  planned.  (6)  (7)  (23)  (25)  (27)  (29) 

•  Saturated  radios.  (9)  (10)  (12)  (13)  (24)  (27)  (34)  (36) 

•  Misleading  radar  calls  which  hurts  situational  awareness  (e.g.,  calling  threats  off  bulls-eye 
point  vice  flight  position).  (1) 

•  Poor  situational  awareness.  (2) 

•  Lack  of  confidence/security/experience.  (3) 

•  Missed  radio  calls  between  flight  members.  (3) 

•  Voice  recognition  and  nicknames  sometimes  help  in  task  saturation.  (4) 

•  Similar  call  signs.  (4) 

•  Lack  of  flexibility  when  changing  a  plan.  (6) 

•  Not  knowing  the  other  members  of  the  flight's  voices.  (6) 

•  Different  people  just  sometimes  have  “bad  days.”  (6) 

•  Missed  communication  (UHF/VHF  interference).  (12) 

•  Someone  in  flight  loses  a  visual  or  radar  contact.  (25) 

•  Using  the  wrong  bulls-eye  to  call-out  threat.  (26) 

•  Communication  loss.  (7) 

•  Not  getting  what  you  want  across.  (7) 

•  Someone  doing  something  different  than  they  are  saying.  (10) 

•  When  people  are  not  concise  in  what  they  say  and  use  too  many  words.  (21) 

•  Flight  lead  who  is  not  calm  on  the  radio.  (30) 

•  Wingmen  who  are  not  calm  on  the  radio.  (30) 

•  Wingmen  not  understanding  their  responsibilities.  (8) 

•  Communication  by  sender  not  clear  to  the  receivers.  (13) 

•  People  who  do  not  keep  up  with  tactical  changes  by  insisting  that:  “this  is  the  way  we’ve 
always  done  it....”  (19) 

•  People  may  not  be  “perceiving”  a  situation  the  same.  (22)  (33) 

•  Not  using  plain  English  when  it  is  needed.  (34) 

•  When  person  with  emergency  has  a  low  experience  level.  (29) 

•  When  the  traffic  pattern  is  busy.  (29) 

•  Unstated  goals  and  objectives  in  the  briefing.  (32) 

•  When  people  walk  out  of  a  briefing  with  unanswered  (or  unasked)  questions.  (32) 

•  Ambiguous  briefing  by  the  flight  lead.  (33) 

•  A  distraction  in  the  aircraft,  like,  dropped  a  map,  cannot  set  radio  frequency,  etc.  (34) 
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Types  of  factors  that  cause  a  communication  breakdown  with  other  flights  or  controlling 
agencies 


•  When  things  don’t  go  as  planned.  (1)  (5)  (14)  (19)  (33) 

•  Lack  of  situational  awareness.  (1)  (4) 

•  Equipment  problems.  (1)  (2)  (4)  (8)  (10)  (21)  (23)  (29)  (32) 

•  Improper  terminology.  (2)  (5)  (6)  (8)  (21)  (22)  (36) 

•  Poor  radio  discipline.  (3)  (6)  (8)  (9)  (12)  (1 8)  (22)  (24)  (25)  (26)  (27) 

•  Lack  of  experience.  (6)  (7) 

•  Not  speaking  up  when  you  should.  (14)  (26) 

•  Saturated  radios.  (20)  (27)  (34)  (35) 

•  False  assumptions.  (25)"  (27) 

•  When  GCI/AWACS  creates  questionable  situation,  additional  communication  is  needed  to 
re-sort  the  tactical  picture.  (1) 

•  What  was  said  was  not  what  was  received  (GCI/Strike  force  interpretation).  (2) 

•  Language  barriers  (between  different  countries).  (4) 

•  During  communication  between  different  fighter  communities,  services,  or  countries,  not 
everything  said  means  the  same  thing  to  everyone  else.  (6) 

•  “Close  sounding  call  signs.”  (12) 

•  People  focusing  only  on  their  own  problems.  (14) 

•  Lack  of  standard  communication  at  a  critical  point.  (25) 

•  Not  using  plain  English.  (25) 

•  A  player  using  nonstandard  communication.  (26) 

•  Not  having  a  well-briefed  communications  plan  between  the  different  players.  (31) 

•  Controlling  agency  not  understanding  what  you  need.  (9) 

•  Rivalry  between  over  aircraft/flights  for  ATC  attention.  (9) 

•  Both  UHF  and  VHF  transmissions  at  the  same  time.  (10) 

•  Lost  situational  awareness  from  either  direction.  (11) 

•  Different  cultures,  like  during  Desert  Storm.  (21) 

•  Miscommunications  with  ATC.  (30) 

•  Players  not  knowing  “what  to  say.”  (13) 

•  Complexity  of  the  plan:  “complex  plans  work  one  time  out  of  ten.”  (19) 

•  Misunderstanding  the  communication  plan  and  the  overall  game  plan.  (20) 

•  When  someone  mislabels  a  group.  (29) 

•  When  someone  says  something  stupid  that  “dumps  everyone  else’s  situational  awareness. 
(29) 

•  Not  all  the  players  know  the  plan.  (32) 

•  Briefing  must  cover  all  the  contingencies  for  all  of  the  assets.  (32) 

•  Radio  out  assets  should  be  sent  home.  (32) 

•  Threats.  (33) 

•  Stress.  (33) 

•  People  not  being  where  they  are  supposed  to  be.  (33) 
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•  FIXES:  Use  victor  frequency  for  internal  flight  communication  and  UHF  for  controlling 
agency  or  package  communications;  think  first,  then  say  it;  use  plain  English;  listen  to  what 
is  going;  and  keep  sight  of  lead.  (33) 


Attitudes  on  Situational  Awareness 
What  a  flight  leader  can  do  to  improve  situational  awareness 

•  Good  briefing  and/or  planning.  (1)  (3)  (6)  (7)  (9)  (10)  (24)  (27)  (31)  (33)  (35)  (36) 

•  Good  communication,  keep  everyone  informed.  (1)  (2)  (3)  (5)  (6)  (8)  (10)  (1 1)  (12)  (13)  (14) 
(18)  (21)  (23)  (24)  (25)  (26)  (27)  (29)  (31)  (32)  (33)  (34)  (35) 

•  Keep  the  plan  simple.  (2)  (20) 

•  Fly  the  plan  as  briefed.  (8)  (9)  (10)  (12)  (19)  (36) 

•  Offer  information  as  briefed,  without  overloading  communication.  (1) 

•  Take  time  to  focus  attention  so  information  can  “sink-in.”  (2) 

•  Attention  to  details.  (2) 

.  Use  JTIDS,  AW  ACS,  ELINT  information,  and  Joint  STARS.  (4) 

•  Worst  Problem:  When  some  people  (like  intel)  know  something,  but  do  not  pass  it  on  to  the 
people  executing  the  mission.  (4) 

•  Use  Data  Link.  (4)  (14) 

•  Keep  the  big  picture  through  target  sort,  visual,  and  listening  to  radio  calls.  (6)  (12)  (26) 

•  Make  sure  everyone  knows  the  priorities  for  each  mission  segment.  (6) 

•  Do  not  overload  inexperienced  wingmen  with  too  much  information.  (14) 

•  When  the  briefed  plan  is  breaking  down,  lead  must  tell  the  rest  of  the  flight  what  the  new 
plan  is.  (24) 

•  If  a  pilot  breaks  a  habit  pattern,  go  back  to  the  last  point  before  the  pattern  was  broken.  (25) 

•  Know  the  contingencies  for  flight/element  split  up.  (26) 

•  “75%  of  situational  awareness  is  knowing  what  is  going  on.”  (27) 

•  Minimize  the  flight  member  taskings  so  that  they  have  enough  time  to  focus  on  what  they 
will  have  to  do.  Keep  the  plan  simple.  (27). 

•  Know  what  the  picture  is  going  to  look  like  on  the  radar  and  where  everyone  is  going  to  be. 

(31) 

•  Know  what  the  altitude  and  the  timing  deconfliction  is.  (31) 

•  Everyone  needs  to  what  the  plan  is  and  why  the  plan  is  the  way  it  is  so  when  things  pop-up 
and  you  have  to  be  flexible,  you  can  anticipate  what  the  flight  lead  may  want  to  do.  (5) 

•  Fly  with  the  same  people  whenever  possible,  so  people  think  the  same.  (5) 

•  Understand  what  you’re  doing  before  you  go  fly.  (7) 

•  Stay  ahead  of  what  you’re  doing.  (7) 

•  Keep  the  big  picture.  (9)  (10) 

•  Help  build  the  big  picture  for  everyone  in  flight.  ( 1 0) 

•  Give  directions  to  keep  flight  in  formation.  (10) 

•  Understand  the  plan  (tactics,  execution,  etc.).  (8)  (13) 

•  Following  wingmen  and  flight  lead  contracts.  (8) 

•  Anticipate  what  is  going  to  happen  and  communicate  it  to  the  rest  of  the  flight.  (29) 
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•  Experienced  flight  members.  (8) 

•  If  the  flight  lead  can  tell  that  a  wingman  went  “tumbleweed,”  he  needs  to  tell  the  wingman 
what  to  do  next  to  get  back  to  the  plan.  (8) 

•  Be  “brief’  on  the  radio  when  building  situational  awareness.  (13) 

•  Delegate  search  responsibilities.  (13) 

•  Get  the  big  picture  and  communicate  it  to  flight.  (13)  (34) 

•  Get  data-link  for  all  information,  from  wingman  fuel  to  location  of  all  players.  (13) 

•  Good  use  of  radar  search  and  air-to-air  TACAN.  ( 1 9) 

•  If  radio  calls  indicate  that  flight  is  losing  situational  awareness,  back  off,  or  knock-it-off,  and 
let  everyone  catch  up.  (19) 

•  Listen  to  what  is  being  said  to  build  the  “big  picture.”  (20) 

•  “Take  your  time  and  get  there  faster.”  (23) 

•  Build  the  picture  using  radar/visual  and  communicate  the  picture  to  flight  on  VHF.  (33) 

•  “Funny  thing  about  situational  awareness,  you  don’t  know  that  you  have  lost  it  until  you  get 
it  back.”  (33) 

•  At  night  or  in  weather,  put  flight  in  radar  trail  if  needed  to  give  them  time  to  build  their 
situational  awareness.  (33) 

•  Verbally  help  wingman  with  radar  work  if  needed.  (34) 

•  Give  the  wingman  a  verbal  description  of  what  his  aircraft  is  doing  if  he  is  disoriented.  (34) 

•  Adapt  the  plan  if  needed.  Give  audible  changes  and  allow  the  wingmen  to  “reconfirm”  if 
they  have  any  questions.  (36) 

How  can  situational  awareness  be  improved  in  a  multiforce  package? 

•  Really  needed  after  a  threat  reaction.  (19) 

•  Use  radio  communication  to  rebuild  situational  awareness  (bulls-eye  calls,  next  steer  point, 
etc.).  (19) 

•  Use  a  simple  game  plan  that  is  understood  by  all  agencies  (ATC,  GCA,  etc.)  and  executable 
by  everyone  involved.  (20) 

•  Everyone  up  on  a  single-strike  frequency  (iron-haulers,  air-to-air,  and  AW  ACS.  (20) 

•  Use  AWACS  as  a  communication  bridge  if  needed  to  communicate  with  other  players,  but 
realize  that  it  will  increase  time  delay.  (20) 

•  Using  data-link  will  speed  things  up.  (20) 

•  “Keep  your  mouth  shut  and  listen.”  (23) 

•  Limit  communication  to  prevent  “trashing  everyone’s’  situational  awareness”  by  hogging  the 
frequency.  Use  victor  for  interflight  communication.  (34) 

•  Stay  with  scripted  maneuvers  and  try  not  to  call  audibles.  (29) 

•  Communicate  to  the  rest  of  the  package  if  you  have  to  pull  out  of  the  “train”  if  it’s  not  part  of 
the  script,  then  call  when  you  are  back  in  position.  (29) 

•  Call  out  briefed  and  nonbriefed  threats  to  the  rest  of  the  package,  even  if  they  are  not  a  threat 
to  your  flight.  (29)  (32) 

•  If  the  package  loses  sight  of  the  big  picture,  then  someone  needs  to  speak  out.  (33) 

•  The  package  commander  must  provide  the  picture  and  listen  up  to  check-ins  to  make  sure  all 
of  his  players  are  there.  (33) 

•  Four-ships  must  be  where  they  were  told  to  be  so  that  everyone  else  has  the  big  picture.  (33) 
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•  So  much  comm  chatter  decreases  the  situational  awareness.  Focus  on  the  game  plan  and 
execute.  (36) 

•  If  deviating  from  the  game  plan,  broadcast  it  (route,  timing,  etc.)  on  UHF,  if  you  can  make  a 
transmission,  then  get  back  to  the  game  plan  as  soon  as  possible.  (36) 

•  Talk  to  your  flight  only  on  Victor.  (36) 

•  A  flight  has  to  stick  together  -  always.  (36) 


What  can  an  individual  pilot  do  to  improve  situational  awareness? 

•  Listen  and  build  a  mental  picture.  (1)  (3)  (1 1)  (12)  (14)  (20)  (23)  (30) 

•  Prepare  (study,  chair  fly,  etc.).  (2)  (21) 

•  Stay  visual  and/or  in  position.  (3)  (27)  (31)  (33)  d 

•  Prioritize  tasks.  (3)  (4)  (5)  (1 1)  (13)  (14)  (18)  (19)  (24)  (27)  (29) 

•  Be  willing  to  speak  up.  (12)  (25)  (29)  (31)  (33) 

•  “Get  excited  about  your  role.”  (2) 

•  Ask  a  question  if  you  are  not  sure.  (12) 

•  Have  bullseye  set  to  the  nearest  divert  unless  needed  for  air-to-air  engagements.  (31) 

•  Know  what’s  going  to  happen  before  you  step  out  the  door  to  fly.  (31) 

•  “Always  think  ahead;  don’t  just  put  out  the  fire-of-the-second.”  (5) 

•  Keep  ahead  of  the  jet  and  review  what  is  ahead.  (10)  (30) 

•  Have  a  game  plan  and  good  habit  patterns.  (5) 

•  Know  what  was  briefed  for  the  plan  and  understand  it.  (9)  (10) 

•  Keep  the  big  picture.  (9) 

•  Do  not  get  focus  fixed  on  any  one  thing.  (10) 

•  Have  a  good  attitude;  be  happy  and  unstressed.  (10) 

•  Use  all  of  the  equipment  available.  (1 1) 

•  “You  do  not  know  that  you  have  lost  situational  awareness  until  you  get  some  back.”  (1 8) 

•  Go  back  to  basics  and  keep  it  simple.  (18) 

•  Do  not  just  fly  your  jet.  If  you  are  always  “backing  up  the  flight  lead,”  you  will  have 
situational  awareness.  (21) 

•  Use  Data  Link  and  Moving  Maps.  (35) 

•  Be  ready  to  go  when  you  “push.”  Verbal  communication  breaks  down  when  the  engagement 
starts.  (35) 

•  Constantly  ask  yourself  about  what  is  happening  and  what  lead  might  do  next.  Then  analyze 
why  things  did  not  happen  the  way  you  thought.  (35) 

•  Fall  back  to  the  plan  in  the  briefing.  (8) 

•  Fly  airplane  and  formation  first.  (13)  (29) 

•  Build  situational  awareness  by  moving  away  from  the  situation  if  necessary  (horizontal  and 
vertical  space),  using  priorities,  radios,  and  radar.  (19) 

•  Be  ready  to  take  over  lead  at  anytime.  Do  not  get  lazy  on  the  wing.  (20) 

•  Build  a  picture  from  “outside  the  cockpit”  to  “inside  the  cockpit.”  (32) 

•  Look  at  the  game  plan  and  the  tactic.  (32) 

•  Cockpit  organization:  where  do  you  put  things  and  what  do  you  write  on,  etc.  (33) 
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•  Keep  up  with  the  steer  points.  (34) 

•  Use  big  visual  references  because  you  probably  will  not  have  time  to  read  a  map.  (34) 

Attitudes  on  characteristics  of  effective  flight  leaders 

•  Good  overall  leader.  (1)  (14)  (33) 

•  Has  good  knowledge  about  aircraft  and  tactics.  (1)  (3)  (5)  (13)  (18)  (21)  (24)  (30)  (33) 

•  Organized.  (1)  (2)  (33) 

•  Has  a  good  personality,  gets  along  well  with  others.  (1)  (5)  (7)  (9) 

•  Has  a  lot  of  charisma.  (2)  (3)  (5)  (9)  (10)  (18) 

•  Good  overall  communicator.  (2)  (4)  (6)  (7)  (12)  (19)  (20)  (26)  (30)  (32)  (35) 

•  Has  good  situational  awareness.  (6)  (7)  (24) 

•  Good  briefer/debriefer.  (4)  (9)  (12)  (13)  (19)  (20)  (26)  (33)  (35) 

•  Encourages  flight  members  to  speak  up.  (9)  (27)  (29)  (30)  (31)  (35) 

•  Stays  ahead  of  jet,  because  he  is  making  decisions  in  advance.  (1) 

•  Selfless;  focused  on  the  mission.  (2) 

•  Proactive.  (3) 

•  Honest  when  he  does  something  wrong.  (3) 

•  Analyzes  the  flight  members  and  gives  a  good  mission  briefing  which  sets  the  tone  for  the 
flight.  (24) 

•  Fly  together  frequently  and  build  a  team  where  people  “think  alike”  (like  ANG  pilots).  (4) 

•  Good  judgement.  (6) 

•  Initiative  and  aggressiveness  to  make  decisions.  (6) 

•  Good  prioritizer.  (6) 

•  Good  task  manager.  (6) 

•  Good  wingman  consideration.  Has  been  a  good  wingman,  so  he  understands  wingmen 
needs.  (12) 

•  Not  necessary  to  be  the  best  pilot,  as  long  as  they  can  brief  and  fly  well.  (12) 

•  Fly  what  is  briefed.  (12) 

•  Delegates  well.  (14) 

•  Never  stops  learning.  (25) 

•  Thinks  for  flight,  not  just  himself.  (25) 

•  Provides  flexibility  for  wingmen  to  speak  up.  (26) 

•  Be  decisive;  make  quick,  confident  decisions  and  stick  to  them.  (3 1) 

•  Be  directive  to  the  rest  of  the  flight.  (3 1 ) 

•  Motivated.  (5) 

•  Everything  happens  in  accordance  with  the  plan.  (7) 

•  “Leaders  who  are  behind  the  power  curve  are  not  respected.  (7) 

•  Recognizes  how  rest  of  flight  is  doing  and  offers  helpful  corrections.  (10) 

•  Willing  to  point  out  and  debrief  his  own  faults,  as  well  as  the  rest  of  the  flights’  .(10) 

•  Creative  to  find  different  ways  to  accomplish  the  mission;  thinks  outside  the  box.  (18) 

•  Knows  how  to  fly  his  jet.  (2 1 ) 

•  Know  everyone’s  limitations  and  then  tailor  the  mission  to  be  within  the  confines  of 
everyone’ s  abilities.  (21) 
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•  Has  a  calm  attitude.  (30) 

•  Can  manage  the  tasks  required.  (8) 

•  Thinks  ahead.  Has  forward  vision  to  see  where  errors  might  occur  and  can  minimize.  (8) 
(13) 

•  Remembers  his  priorities:  aviate,  navigate,  communication.  (13) 

•  Must  understand  the  limitations  of  the  guys  in  the  flight.  (20) 

•  Does  not  have  to  be  the  world’s  great  stick,  but  must  be  able  to  assimilate  a  lot  of 
information  and  make  good  decisions.  (20) 

•  Uses  the  KISS  principle.  (20) 

•  Keeps  track  of  the  execution  and  the  contingencies  ( weather,  etc.).  (20) 

•  Technically  competent.  (23) 

•  Confident  so  that  wingmen  will  trust  lead.  (23)  (29) 

•  Methodical,  logical,  and  predicable,  so  that  wingmen  can  figure  out  what  lead  is  going  to  do 
next.  (29) 

•  Can  accomplish  quality  training  without  “wasting  the  flight  members’  day  or  beating  the 
subject  to  death.”  (19) 

•  Looks  for  the  90%  solution  to  a  problem.  (19) 

•  Confident.  (33) 

•  Focused  and  knows  what  is  going  on.  (33) 

•  Always  has  smooth  flights.  (33) 

•  Knows  the  experience  level,  currency,  and  needs  of  the  people  in  the  flight,  so  he  can 
determine  what  to  accomplish.  (34) 

•  Maintains  positive  flight  lead  control,  while  still  encouraging  wingmen  to  think  for 
themselves.  (34) 

Attitudes  on  characteristics  of  effective  wingmen 


•  Maintains  “contract”  in  the  air  (position,  visual,  radios,  radar).  (1)  (6)  (7)  (9)  (10)  (1 1)  (13) 
(24)  (25)  (27)  (29)  (30)  (35) 

•  Stays  ahead  of  things.  (1)  (14)  (24) 

•  Good  listener.  (2)  (3)  (12)  (24)  (32)  (34) 

•  Can  take  constructive  criticism.  (3)  (24) 

•  Knowledgeable.  (3)  (14) 

•  Good  follower.  (6)  (12)  (23) 

•  Willing  to  speak  up  when  necessary.  (5)  (6)  (8)  (12)  (14)  (21)  (24)  (26)  (27)  (30)  (34)  (35) 

•  Motivated  to  learn  and  do  well.  (1 0)  (1 8)  (23)  (31)  (33)  (34) 

•  Good  situational  awareness.  (7)  (10)  (29)  (30) 

•  Knows  his  role  and  position  in  the  flight.  (1)  (2) 

•  Thorough;  thinks  through  the  entire  mission.  (2) 

•  Confident.  (2) 

•  Aggressive  (“but  that’s  a  fine  line....”).  (3) 

•  Good  task  prioritization — knows  what  is  important.  (4)  (6) 

•  Will  fly  the  plan.  (12) 

•  Not  worried  about  what  lead  is  going  to  think  or  say.  (12) 


83 


•  Ready  to  fly.  (24) 

•  Ready  to  assist  the  flight  lead.  (24) 

•  Backs  up  the  flight  leader.  (25) 

•  Does  not  say  anything  until  it  is  really  needed.  (25) 

•  Do  what  you  are  told  to  do.  (5) 

•  Do  your  tasks  well  so  that  the  flight  lead  will  trust  you  and  not  have  to  worry  about  you.  (5) 

•  Someone  who  does  not  talk  all  the  time,  but  does  when  it  is  important  to  talk.  (5)  (21) 

•  Do  not  be  a  blind  follower;  help  with  rest  of  four-ship.  (7) 

•  Backup  flight  lead/element  lead.  (9) 

•  “Keep  focus  on  the  big  picture,  not  just  looking  through  a  soda-straw.”(9) 

•  Be  a  help,  not  a  hindrance.”  (10) 

•  Knows  what’s  going  on  and  how  to  fly  the  airplane.  (11) 

•  Tries  to  think  what  the  flight  lead  is  thinking  about.  (30) 

•  Understands  the  plan  and  is  able  to  execute  the  plan,  or  communicate  it  if  he  cannot.  (8) 

•  Someone  who  continues  to  fly  as  if  they  are  still  being  graded,  even  after  their  MR  check. 

(20) 

•  Someone  who  is  not  intimidated  by  a  flight  lead.  (20) 

•  Someone  who  gets  into  the  books  and  asks  questions.  (20) 

•  Technically  competent  at  basic  level.  (23) 

•  Has  qualities  to  be  a  good  flight  leader.  (33) 

•  Remembers  the  contracts  and  executes  them.  (34) 
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